Making Software

Not All Bug Reports Get Fixed

Most projects have more bug reports that can be fixed given limited resources and time. To characterize how the quality of bug reports increases the chances of bugs getting fixed, we sampled 150,000 bugs from Apache, Eclipse, and Mozilla (50,000 per project). These bugs had various resolutions, such as FIXED, DUPLICATE, MOVED, WONTFIX, and WORKSFORME. We divided the bug reports into two groups, successful and not successful, and then used statistical tests such as Chi-Square and Kruskal-Wallis (p<.05) to check for relationships between the success of bug reports and the presence of information items (code samples, stack traces, patches, screenshots). Our comparisons were:

Resolution of bug reports

We compared bug reports resolved as FIXED against bug reports with other resolutions, such as WONTFIX and WORKSFORME. We treated duplicate bug reports as a separate group because for some the master report is fixed, whereas for others the master report is not fixed.

Lifetime of bug reports

We compared bug reports with a short lifetime against bug reports with a long lifetime. A fast turnaround for bug reports is desirable, especially for users, but also for developers.

We also checked for the influence of readability on the success of bug reports. To measure the readability of bug reports, we used the Style tool, which “analyses the surface characteristics of the writing style of a document” [Cherry and Vesterman 1981]. The readability of a text is measured by ...