ArticlePDF Available

An empirical study of supplementary bug fixes

Authors:

Abstract and Figures

A recent study finds that errors of omission are harder for programmers to detect than errors of commission. While several change recommendation systems already exist to prevent or reduce omission errors during software development, there have been very few studies on why errors of omission occur in practice and how such errors could be prevented. In order to understand the characteristics of omission errors, this paper investigates a group of bugs that were fixed more than once in open source projects — those bugs whose initial patches were later considered incomplete and to which programmers applied supplementary patches. Our study on Eclipse JDT core, Eclipse SWT, and Mozilla shows that a significant portion of resolved bugs (22% to 33%) involves more than one fix attempt. Our manual inspection shows that the causes of omission errors are diverse, including missed porting changes, incorrect handling of conditional statements, or incomplete refactorings, etc. While many consider that missed updates to code clones often lead to omission errors, only a very small portion of supplementary patches (12% in JDT, 25% in SWT, and 9% in Mozilla) have a content similar to their initial patches. This implies that supplementary change locations cannot be predicted by code clone analysis alone. Furthermore, 14% to 15% of files in supplementary patches are beyond the scope of immediate neighbors of their initial patch locations — they did not overlap with the initial patch locations nor had direct structural dependencies on them (e.g. calls, accesses, subtyping relations, etc.). These results call for new types of omission error prevention approaches that complement existing change recommendation systems.
Content may be subject to copyright.
An Empirical Study of Supplementary Bug Fixes
Jihun Park,
Miryung Kim,
Baishakhi Ray,
Doo-Hwan Bae
Korea Advanced Institute of Science and Technology
{jhpark, bae}@se.kaist.ac.kr
The University of Texas at Austin
{miryung@ece., rayb@}utexas.edu
Abstract—A recent study finds that errors of omission are
harder for programmers to detect than errors of commis-
sion. While several change recommendation systems already
exist to prevent or reduce omission errors during software
development, there have been very few studies on why errors
of omission occur in practice and how such errors could
be prevented. In order to understand the characteristics of
omission errors, this paper investigates a group of bugs that
were fixed more than once in open source projects—those bugs
whose initial patches were later considered incomplete and to
which programmers applied supplementary patches.
Our study on Eclipse JDT core, Eclipse SWT, and Mozilla
shows that a significant portion of resolved bugs (22% to 33%)
involves more than one fix attempt. Our manual inspection
shows that the causes of omission errors are diverse, including
missed porting changes, incorrect handling of conditional state-
ments, or incomplete refactorings, etc. While many consider
that missed updates to code clones often lead to omission
errors, only a very small portion of supplementary patches
(15% in JDT, 17% in SWT, and 8% in Mozilla) have a content
similar to their initial patches. This implies that supplementary
change locations cannot be predicted by code clone analysis
alone. Furthermore, 14% to 15% of files in supplementary
patches are beyond the scope of immediate neighbors of their
initial patch locations—they did not overlap with the initial
patch locations nor had direct structural dependencies on them
(e.g. calls, accesses, subtyping relations, etc.). These results call
for new types of omission error prevention approaches that
complement existing change recommendation systems.
Keywords-software evolution; empirical study; patches; bug
fixes
I. INTRODUCTION
According a recent study, developers are over ve times
more accurate at detecting errors of commission than errors
of omission [1]. To prevent or reduce omission errors,
several change recommendation systems [2]–[6] suggest
additional change locations given a change set to reduce
inconsistent or missing updates. These change recommenda-
tion systems make their own assumptions about which types
of omission errors are common and how such errors could be
reduced. For example, FixWizard suggests code clones of an
existing bug fix to reduce potential missed updates [3]. As
another example, Robillard uses the dependency structure
of a change set to suggest where additional changes need
to be made [4]. Zimmermann et al. and Ying et al. predict
additional change locations based on historical co-change
patterns derived from version histories [6], [7]. Hassan
and Holt predict additional change locations based on co-
change patterns and the dependency-graph of a change set
in conjunction [2].
While these studies make various assumptions about how
additional change locations can be recommended, there have
been very few studies on how often and why incomplete
fixes occur in practice. In this paper, we investigate the ex-
tent and characteristics of supplementary patches—patches
that were later applied to supplement or correct initial fix
attempts. We investigate the percentage of bugs that require
supplementary fixes, their severity levels, and the relation-
ship between initial patches and supplementary patches in
terms of location and content similarity using the version
history of Eclipse JDT core, Eclipse SWT, and Mozilla.
The following summarizes our study findings.
How many bugs require supplementary patches? A
considerable portion of resolved bugs (22% in Eclipse
JDT core, 24% in Eclipse SWT, and 33% in Mozilla)
involve supplementary patches. The bugs with supple-
mentary patches are more likely to have higher sever-
ity—they have 35%, 64%, and 13% higher proportion
of the Blocker/Critical status, take 56%, 91%, and 66%
longer to be resolved, and involve 21%, 37%, and 62%
more developers in bug discussions.
What are the common causes of incomplete fixes?
We randomly sampled total one hundred supplementary
patches and inspected their content and correspond-
ing initial patches. Our manual inspection shows that
the common causes of omission errors are diverse,
including missed porting changes, incorrect handling
of conditional statements, and incomplete refactorings,
etc. Furthermore, we found that the size and number
of files of incomplete patches are larger than those
of regular patches and that the file-level dispersion
of incomplete patches is higher than that of regular
patches. These results imply that incomplete fixes tend
to be larger and more scattered than regular fixes.
Are supplementary patches similar to corresponding
initial patches? Several approaches attempt to reduce
omission errors by identifying missed or inconsistent
updates to similar code fragments [3], [8], [9]. To esti-
mate the utility of such techniques in reducing omission
errors, we compared the content of an initial patch
and its supplementary patches using CCFinder [10]. We
found that only 15%, 17%, and 8% of supplementary
patches have a content that is at least 5 lines similar
to its initial patch in Eclipse JDT core, Eclipse SWT,
and Mozilla respectively. This indicates that predicting
a supplementary fix location using code clone analysis
alone is insufficient.
Where is the location of supplementary patches with
respect to initial patches?
48% and 42% of files in a supplementary patch involve
changes within 25 lines of the initial patch location
in Eclipse JDT core and Eclipse SWT respectively.
Furthermore, 32% and 29% of files in supplementary
patches have direct structural dependence (e.g., direct
calls, accesses, and subtyping relations) on the corre-
sponding initial patches. However, the remaining 15%
and 14% of files in a supplementary patch do not have
direct dependence to nor overlap with the initial patch
location.
These results call for new type of omission error pre-
vention approach that complements existing change recom-
mendation systems, which are based on clone detection or
structural dependence analysis.
The rest of this paper is organized as follows. Section II
introduces related work, and Section III describes our study
subjects and analysis method. Section IV presents empirical
results, Section V discusses threats to validity, and Sec-
tion VI summarizes our study.
II. RELATED WORK
Errors of Omission. Fry and Weimer studied how well a
human can localize different types of bugs [1]. They found
that humans are over five times more accurate at locating ex-
tra statements than missing statements. This result is aligned
with the fact that the omission errors are much more difficult
to find than commission errors. While Fry and Weimer focus
on measuring the accuracy of fault localization for different
types of defects, our study focuses on the characteristics of
supplementary patches to derive an insight into the common
causes and characteristics of omission errors.
Nguyen et al. find that 17% to 45% of total fixes are recur-
ring bug fixes [3]. Their tool FixWizard suggests additional
change locations by finding code clones of an existing bug
fix. Our study find that only 9% to 26% of supplementary
patches have a content that is at least five lines similar to
their initial patches, indicating that clone analysis alone is
inadequate for reducing incomplete fixes.
Kim et al. suggest an approach to determine whether
null-dereference bug fixes are complete or incomplete, with
the concept of bug neighborhood [11]. A bug neighbor-
hood refers to program statements directly related to null-
dereference bugs. Zhang et al. propose a method to detect
execution omission errors using dynamic slicing [12]. While
these techniques focus on detecting omission errors, our
study investigates the extent and characteristics of omission
errors using the history of incomplete and supplementary
bug fixes found in version history.
Empirical Studies of the Extent of Supplementary Bug
Fixes. Yin et al. investigated incorrect bug fixes in large
operating systems [13]. They found that 14.8% to 24.4% of
post release patches are incorrect and that the most difficult
type of bugs are related to concurrency. Their results show
that a considerable portion of bugs was fixed more than
once. Gu et al. measured the number of reopened bug reports
in the bug database of Ant, AspectJ, and Rhino and found
that such incorrect fixes correspond to as much as 9% of
all bugs [14]. They assessed bug fixes with two criteria
coverage—if the fix correctly handles all inputs triggering
a bug, and disruption—if the fix unintentionally changes
intended behavior. Purushothaman et al. investigated the
extent of small changes and found that nearly 40% of bug
fixes cause one or more defects in Lucent 5ESS. These
results are aligned with our study that incomplete bug fixes
are common in practice [15].
Change Recommendation Systems for Supplementary
Fixes. To reduce omission errors, Robillard’s approach [4]
takes a change set as input and recommends additional
change locations based on the dependence structure of the
change set. His approach is based on the assumption that
additional change locations are likely to have structural
dependencies on already changed code. Hassan and Holt
propose several change propagation heuristics and find that
historical change coupling is a more accurate predictor
than structural dependencies such as method call relation-
ships [2]. Padioleau et al. [16] find that Linux device
drivers often co-evolve when kernel APIs change, and
suggest an approach that infers a generic patch from an
example edit [17]. Based on the similar assumption, Wang
et al.s approach automatically find edit locations similar to
an existing bug fix using dependence-related queries [18].
While these approaches make individual assumptions about
how supplementary fix locations can be predicted based on
the content and location of an existing patch, our study
investigates the location, content, and characteristics of
incomplete and supplementary patches.
III. STUDY APPROACH
This section describes our subject programs and the
method of identifying supplementary bug fixes.
Study Subjects. We select Eclipse JDT core, Eclipse
SWT and Mozilla as our study subjects. Eclipse is an
integrated development environment, which has been widely
used in various mining software repositories research.
Eclipse JDT core and Eclipse SWT are sub-projects of
Eclipse written in Java. Mozilla is the open source project
for the web. Mozilla contains many different sub-projects,
e.g., Firefox web browser, Thunderbird mail client, etc.
Identification of Supplementary Bug Fixes. We study the
evolution period of 2001/06 to 2009/02 in Eclipse JDT core,
2001/05 to 2010/05 in Eclipse SWT, and 1998/03 to 2008/05
in Mozilla. Previous studies on bug fixes [19], [20] found
that it is possible to extract bug fix revisions by matching
keywords against commit logs using the heuristic developed
by Mockus and Votta [21]. We parse change logs to look
for bug ID numbers using #().,\n\t\r\f as a delimiter and
regard all integers as potential bug IDs. We then check
those numbers against the corresponding bug databases
to ensure that those extracted numbers indeed correspond
to bug IDs. All of them use CVS as their configuration
management system. In order to group file-revisions checked
in the same commit, we convert CVS repositories to SVN
repositories using cvs2svn.
1
Using this method, we identify
that 30.59%, 31.44%, and 34.56% of commits are bug fixes
in Eclipse JDT core, Eclipse SWT, and Mozilla respectively.
We define supplementary bug fixes as commits that con-
tain a bug ID that was previously fixed. For example, when
the same bug ID appears more than once in a version history,
we consider all but the first commit as supplementary fixes
for each closed bug ID. For example, the revision 8998
is an initial patch and the revisions 9012 and 9014 are
supplementary patches in Table I.
Table I
TWO SUPPLEMENTARY PATCHES ARE APPLIED TO FIX BUG # 52916.
Revision Date Author Comment
8998 2004/07/07 kjohnson 52916
8999 2004/07/07 kjohnson *** empty log message ***
9000 2004/07/07 othomann HEAD - fix for 69349
9001 2004/07/07 othomann HEAD - Fix for 69271
... ... ... ...
9011 2004/07/08 pmulet 69375
9012 2004/07/08 kjohnson 52916
9013 2004/07/08 pmulet *** empty log message ***
9014 2004/07/08 kjohnson 52916 in 3.1 stream
To make sure that bugs are completely resolved and will
not be re-opened later, we consider only the bug reports
reported within the period of 2004/07 to 2006/07 in Eclipse
JDT core, 2004/07 to 2006/07 in Eclipse SWT, and 2003/04
to 2005/07 in Mozilla, when extracting information from
bug databases.
After identifying bug IDs, we categorize bug fix com-
mits into two categories: (1) initial fix revisions that refer
to first attempts to address a particular bug ID and (2)
supplementary fix revisions that refer to later attempts to
correct, extend, complement, or revert an initial fix. Based
on this categorization, we group bug reports into two groups:
(1) Type I bugs—the bugs that were mentioned in only one
commit and (2) Type II bugs—the bugs that were mentioned
in multiple fix revisions.
1
http://cvs2svn.tigris.org/
IV. RESULTS
Section IV-A investigates the number of bugs with supple-
mentary patches and their severity. Section IV-B describes
the common cause of incomplete bug fixes that we found
through manual inspection of one hundred randomly sam-
pled incomplete patches and corresponding supplementary
patches. Section IV-C investigates whether the content of
supplementary bug fixes is similar to the content of cor-
responding initial bug fixes using clone detection analysis.
Section IV-D investigates the relationship between an initial
patch location and its supplementary patch location.
A. How Many Bugs Require Supplementary Patches?
Table IV-A shows our research subjects and the number
of bugs fixed only once (Type I bugs) vs. the number of bugs
with more than one fix commits (Type II bugs). 22.46% to
32.81% of resolved bugs require supplementary patches. In
Figure IV-A, X axis represents how many times the same bug
ID has been occurred in fix commits, and Y axis represents
the percentage of bugs with n patches out of all Type II
bugs. For example, in the Eclipse JDT core project, more
than 70% of Type II bugs are fixed twice.
Figure IV-A shows the number of days taken for a
supplementary fix to appear since an initial attempt to fix.
If there are multiple fix commits, we measure the time gap
between the first fix revision and the last fix revision. More
than 60% of supplementary fixes in JDT core appear within
24 hours. The majority of supplementary fixes are resolved
in a short amount of time. However, some supplementary
fixes take a very long time and take a large number of fix
attempts. For example, in Mozilla, one bug was fixed 53
times, and one bug report in Eclipse SWT took a total of
1113 days to be resolved.
To investigate the difficulty level of Type I bugs and Type
II bugs, we study the severity of bug reports reported in the
bug database, the time taken to resolve the individual bugs,
and the total number of developers involved in each bug
report. Table III shows the results. Severity levels indicate
how serious a defect is: Blocker/Critical/Major/Normal/Mi-
nor/Trivial/Enhancement in the descending order of sever-
ity. The severity distributions for Type I and Type II bugs
are different. Overall, Type II bugs are 35.26%, 63.78%, and
13.08% more likely to be Blocker or Critical in Eclipse JDT
core, Eclipse SWT, and Mozilla respectively.
Table IV shows the average number of developers in-
volved in the bug discussion and the average time taken to
resolve individual bug. In Eclipse JDT core, 3.67 developers
are involved in discussing Type I bugs, while 4.44 developers
are involved in the discussion of Type II bugs on average.
In Eclipse SWT and Mozilla, 3.13 and 4.50 developers are
involved in Type I bugs respectively, while 4.29 and 7.28
developers are involved in Type II bugs on average. The
time taken to resolve individual bug is measured by the time
taken from REPORTED to FIXED or RESOLVED status.
Table II
STUDY SUBJECTS
Eclipse JDT core Eclipse SWT Mozilla Project
Type IDE IDE Several projects associated with Internet
Period of development 2001/06 2009/02 2001/05 2010/05 1998/03 2008/05
Study period 2004/07 2006/07 2004/07 2006/07 2003/04 2005/07
Total revisions 17000 revisions 21530 revisions 200000 revisions
# of bugs 1812 1256 11254
# of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%)
# of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%)
0%
10%
20%
30%
40%
50%
60%
70%
80%
2 3 4 5 6 7
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
2 3 4 5 6 7 8 9 10 11 12
Percentage
Eclipse SWT
0%
10%
20%
30%
40%
50%
60%
70%
2 4 6 8 10 12 14 16 18 21 25 29
Percentage
Mozilla
Figure 1. The number of times that the same bug is fixed
0%
10%
20%
30%
40%
50%
60%
70%
0
3
6
9
12
16
19
25
29
38
44
52
60
80
97
103
130
166
495
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
70%
0
4
8
16
22
27
35
47
58
73
84
119
172
265
319
428
957
Percentage
Eclipse SWT
0%
5%
10%
15%
20%
25%
30%
35%
0
31
63
94
127
161
196
231
278
336
398
464
531
585
659
733
804
887
1014
Percentage
Mozilla
Figure 2. The number of days taken for the supplementary fix to appear since an initial fix
Table III
SEVERITY OF TYPE I AND TYPE II BUGS
Eclipse JDT core Eclipse SWT Mozilla
Type I bugs Type II bugs Type I bugs Type II bugs Type I bugs Type II bugs
Blocker 1.07% 1.47% 1.99% 2.65% 2.41% 1.79%
Critical 2.56% 3.44% 3.67% 6.62% 8.83% 10.92%
Major 8.33% 8.11% 11.53% 15.23% 8.38% 10.44%
Normal 76.80% 76.90% 74.95% 61.92% 60.92% 61.56%
Minor 5.20% 2.95% 2.41% 1.66% 6.96% 5.39%
Trivial 1.64% 0.00% 1.78% 1.66% 6.87% 3.31%
Enhancement 4.41% 6.88% 3.56% 10.26% 5.61% 6.53%
If a bug is not resolved yet, we measure the time taken from
REPORTED to the latest fix attempt. In Eclipse JDT core,
Type I bugs take 120.79 days and Type II bugs take 188.27
days to be resolved. The others follow similar trends.
Overall, Type II bugs involve more developers in the
bug report discussions and take longer time to be resolved
than Type I bugs (p-values from T-test: 1.45e-12, 1.39e-09,
2.05e-84 for the number of developers in the bug report
discussion, and 3.84e-04, 2.65e-07, and 8.40e-42 for bug
resolution time in Eclipse JDT core, Eclipse SWT, and
Mozilla respectively).
A considerable portion of bugs requires
supplementary patches. Such bugs take longer to be
resolved and involve more developers in the
discussion of the bug reports.
B. What Are The Common Causes of Incomplete Bug Fixes?
To understand why omission errors occur in practice,
we first contrast the characteristics of incomplete patches
(initial patches of Type II bugs) against that of regular
patches (patches of Type I bugs). We measure the average
Table V
THE SIZE AND PHYSICAL DISPERSION OF INCOMPLETE PATCHES VS. REGULAR PATCHES.
Files LOC Added LOC (%) Dispersion (file) Dispersion (package)
Regular Incomplete Regular Incomplete Regular Incomplete Regular Incomplete Regular Incomplete
JDT 2.58 3.91 92.13 203.38 63.73 66.50 0.27 0.28 0.12 0.11
SWT 1.94 2.16 47.58 73.39 65.90 71.39 0.25 0.25 0.11 0.12
Mozilla 3.62 6.21 170.99 340.31 59.76 62.42 0.33 0.37 0.16 0.20
Total 3.30 5.72 147.98 309.38 60.92 63.42 0.31 0.36 0.15 0.18
p-value 1.15E-18 4.46E-05 4.03E-12 2.05E-09 1.04E-10
Table IV
THE EFFORT TAKEN TO FIX TYPE I AND TYPE II BUGS
The number of developers required for fixing bugs
Type I Type II p-value
Eclipse JDT core 3.67 4.44 1.45e-12
Eclipse SWT 3.13 4.29 1.39e-09
Mozilla 4.70 7.28 2.05e-84
The time taken to resolve bugs
Type I Type II p-value
Eclipse JDT core 120.79 188.27 3.84e-04
Eclipse SWT 176.99 337.32 2.65e-07
Mozilla 594.50 805.92 8.40e-42
number of files, the total number of changed lines, the
portion of added lines among all changed lines, and physical
dispersion. To measure the physical dispersion, we compute
normalized entropy scores at the file and package level,
entropy =
P
n
i=1
p
i
log
n
(p
i
), where p
i
is the probability
that a changed line is made to a particular changed file (or
package), using the same method by Hassan [22]. A low
entropy score implies that only a few files include most of
the modifications. If the entropy is high, changed code is
more equally distributed among different changed files. The
results are summarized in Table V. Incomplete patches are
larger in size than regular patches, and they tend to include
more scattered and non-localized edits.
To investigate the common causes of omission errors,
we randomly selected one hundred incomplete patches and
inspected their patch contents, the structural dependence
relations between an incomplete patch and its supplementary
patches, and the associated bug reports. The following
summarizes a taxonomy of common omission errors we
found in the subject programs.
1) An initial patch is ported to a different component
or branch. For example, to fix the bug 88829 in Eclipse
SWT, the patch for Windows is later ported to Mac on
the same file Table.java. Ported edits are usually
identical or very similar to the original edits, but the
content is often adjusted for different target components
or branches as shown in Figure 3.
2) The conditional statement of an initial fix is not
correct. For example, the initial patch of bug 80699
in Eclipse JDT core modified control flows, when the
condition (modifiers && AccAnnotation) is
true. Later, the condition was modified to handle the
corner case of AccInterface as shown in Figure 4.
3) Code elements referring to or being referenced
by changed code (i.e., calls, accesses, or extends) are
later updated. For example, the programmer originally
fixed Display to fix bug 93294 in SWT. The class
Device, a super-type of Display, was later modi-
fied.
4) An initial patch is reverted. For example, an API
added at initial patch of bug 110048 in Eclipse JDT
core was deleted during a supplementary fix because
the API is turned out to be inadequate, and developers
decided to apply a new approach. Its status was changed
to REOPENED.
5) Two different parts calling different subclasses of
the same type are not updated together. The initial
patch of bug 81244 in Eclipse JDT core is about the use
of ArrayTypeReference, and the supplementary
fix is regarding the use of ArrayQualifiedType-
Reference. The two classes are subtypes of the same
super type. (See Figure 5)
6) Incomplete refactoring induces a supplementary
patch. The initial patch of bug 104664 in Eclipse
JDT core refactored file and zipFile functions
partially. The remaining parts are fixed during a sup-
plementary fix. (See Figure 6)
7) The locations of incomplete and supplementary
fixes are related but cannot be checked using the
Java compiler. The initial patch of bug 83593 in
Eclipse JDT core was made to class CopyResource-
ElementsOperation and the supplementary patch
was made to class DOMFinder. The two classes do
not have direct static call dependencies, as the code
uses the Visitor design pattern [23].
8) An initial patch is refactored during
supplementary fixes. The initial patch of
bug 287052 in Mozilla modified the function
CERT_FindCRLReasonExten, which was later
renamed to CERT_FindCRLEntryReasonExten
in the supplementary patch.
9) The comment is improved to explain an initial
patch in detail. For example, the comment for bug
243392 in Mozilla is updated to explain the initial fix
on nsContentSink.cpp.
10) Others. Other omission errors include missed value
initializations, missing null pointer checks, missing
property updates, and forgetting to release run-time
resources. For example, the initial patch of bug 114935
in Eclipse JDT core was on CompilationUnit-
Resolver.java, to exit a loop early, when there
is no need to resolve types further. The remaining
resources were not cleaned up, and this problem was
fixed during the supplementary fix.
Table VI shows the classification of one hundred incom-
plete patches into the above ten categories. 28 out of 100
incomplete patches involve missed porting updates, 23 of
them involve incorrect handling of conditional statements,
and 15 of them involve forgetting to update code that
references modified code, etc.
Table VI
CATEGORIZATION OF 100 SAMPLED INCOMPLETE FIX REVISIONS
# Bug ID (Project)
1) 28 69152 (JDT), 91709 (JDT), 140879 (JDT), 148010 (JDT),
76185 (SWT), 76391 (SWT), 83408 (SWT), 83819 (SWT),
85666 (SWT), 85962 (SWT), 87554 (SWT), 88829 (SWT),
89785 (SWT), 90856 (SWT), 107777 (SWT), 203041
(Mozilla), 203211 (Mozilla), 220933 (Mozilla), 223111
(Mozilla), 224313 (Mozilla), 225424 (Mozilla), 226600
(Mozilla), 231166 (Mozilla), 235859 (Mozilla), 249337
(Mozilla), 261886 (Mozilla), 271855 (Mozilla), 280740
(Mozilla)
2) 23 80699 (JDT), 92315 (JDT), 92888 (JDT), 96763 (JDT),
105531 (JDT), 110422 (JDT), 111494 (JDT), 119844 (JDT),
139621 (JDT), 142772 (JDT), 70318 (SWT), 72401 (SWT),
81081 (SWT), 85386 (SWT), 85867 (SWT), 200144
(Mozilla), 211470 (Mozilla), 212222 (Mozilla), 216581
(Mozilla), 227432 (Mozilla), 232094 (Mozilla), 258278
(Mozilla), 289558 (Mozilla)
3) 15 79772 (JDT), 108372 (JDT), 120264 (JDT), 120640 (JDT),
123522 (JDT), 133071 (JDT), 76750 (SWT), 81987 (SWT),
83543 (SWT), 88463 (SWT), 90024 (SWT), 92230 (SWT),
93294 (SWT), 94003 (SWT), 94467 (SWT)
4) 7 98906 (JDT), 110048 (JDT), 141289 (JDT), 85069 (SWT),
246966 (Mozilla), 251589 (Mozilla), 272046 (Mozilla)
5) 4 81244 (JDT), 126180 (JDT), 78554 (SWT), 106289 (SWT)
6) 3 104664 (JDT), 125518 (JDT), 263216 (Mozilla)
7) 2 83593 (JDT), 126625 (JDT)
8) 7 100636 (JDT), 117302 (JDT), 75148 (SWT), 110559
(SWT), 223435 (Mozilla), 287052 (Mozilla), 298429
(Mozilla)
9) 5 86580 (JDT), 110797 (JDT), 130390 (JDT), 243392
(Mozilla), 298756 (Mozilla)
10) 6 114935 (JDT), 215587 (Mozilla), 217149 (Mozilla), 217999
(Mozilla), 225570 (Mozilla), 240550 (Mozilla)
The common causes of incomplete fixes are diverse.
Incomplete patches are larger in size and more
scattered than regular patches.
C. Are Supplementary Bug Fixes Similar to Corresponding
Initial Fixes?
It is widely believed that code clones are difficult to
maintain and that inconsistent management of code clones
An initial patch (revision 10911 in Eclipse SWT)
Index: .../win32/org/eclipse/swt/widgets/Table.java
========================================
@@ -2180,10 +2180,12 @@
OS.InvalidateRect (handle, null, true);
TableColumn[] newColumns = new TableColumn [count];
System.arraycopy (columns, 0, newColumns, 0, count);
+ RECT newRect = new RECT ();}
for (int i=0; i<count; i++) {
- TableColumn column = newColumns [oldOrder [i]];
+ TableColumn column = newColumns [i];
if (!column.isDisposed ()) {
- if (order [i] != oldOrder [i]) {
+ OS.SendMessage (hwndHeader,
OS.HDM_GETITEMRECT, i, newRect);
+ if (newRect.left != oldRects [i].left) {
column.sendEvent (SWT.Move);
}
}
A supplementary patch (revision 10915 in Eclipse SWT)
Index: .../carbon/org/eclipse/swt/widgets/Table.java
========================================
@@ -2047,21 +2047,34 @@
...
TableColumn[] newColumns = new TableColumn
[columnCount];
System.arraycopy (columns, 0, newColumns,
0, columnCount);
for (int i=0; i<columnCount; i++) {
- TableColumn column = newColumns [oldOrder [i]];
+ TableColumn column = newColumns [i];
if (!column.isDisposed ()) {
- if (order [i] != oldOrder [i]) {
+ if (newX [i] != oldX [i]) {
column.sendEvent (SWT.Move);
}
}
Figure 3. An initial patch is ported to a different component or branch.
An initial patch (revision 10181 in Eclipse JDT core)
Index: .../compiler/classfmt/ClassFileReader.java
========================================
@@ -584,6 +584,7 @@
*
/
public int getKind() {
int modifiers = getModifiers();
+ if ((modifiers & AccAnnotation) != 0)
return IGenericType.ANNOTATION_TYPE_DECL;
if ((modifiers & AccInterface) != 0)
return IGenericType.INTERFACE_DECL;
...
A supplementary patch (revision 10189 in Eclipse JDT core)
Index: .../compiler/classfmt/ClassFileReader.java
========================================
@@ -584,16 +584,19 @@
*
/
public int getKind() {
int modifiers = getModifiers();
- if ((modifiers & AccAnnotation) != 0)
return IGenericType.ANNOTATION_TYPE_DECL;
- if ((modifiers & AccInterface) != 0)
return IGenericType.INTERFACE_DECL;
+ if ((modifiers & AccInterface) != 0) {
+ if ((modifiers & AccAnnotation) != 0)
return IGenericType.ANNOTATION_TYPE_DECL;
+ return IGenericType.INTERFACE_DECL;
+ }
...
Figure 4. The conditional statement of an initial fix is not correct.
An initial patch (revision 10584 in Eclipse JDT core)
Index: .../compiler/problem/ProblemReporter.java
========================================
@@ -2895,10 +2895,8 @@
...
} else if (location instanceof ArrayTypeReference) {
- if (!(location instanceof
ParameterizedSingleTypeReference)) {
- ArrayTypeReference arrayTypeReference =
...
+ ArrayTypeReference arrayTypeReference =
(ArrayTypeReference) location;
+ end = arrayTypeReference.originalSourceEnd;
...
A supplementary patch (revision 10585 in Eclipse JDT core)
Index: .../compiler/problem/ProblemReporter.java
========================================
@@ -2877,11 +2877,9 @@
...
} else if (location instanceof
ArrayQualifiedTypeReference) {
- if (!(location instanceof
ParameterizedQualifiedTypeReference)) {
- ArrayQualifiedTypeReference
arrayQualifiedTypeReference =
...
+ ArrayQualifiedTypeReference
arrayQualifiedTypeReference =
(ArrayQualifiedTypeReference) location;
+ long[] positions = arrayQualifiedTypeReference.
sourcePositions;
+ end = (int) positions[positions.length - 1];
...
Figure 5. Two different parts calling different subclasses of the same type
are not updated together.
An initial patch (revision 11796 in Eclipse JDT core)
Index: .../compiler/batch/ClasspathJar.java
========================================
@@ -23,16 +23,17 @@
...
public class ClasspathJar extends ClasspathLocation {
-ZipFile zipFile;
-boolean closeZipFileAtEnd;
-Hashtable packageCache;
+private File file;
+private ZipFile zipFile;
...
public ClasspathJar(File file) throws IOException {
- this(new ZipFile(file), true, null);
+ this(file, true, null);
}
A supplementary patch (revision 11817 in Eclipse JDT core)
Index: .../compiler/batch/ClasspathJar.java
========================================
@@ -88,10 +88,13 @@
...
this.packageCache = null;
}
public String toString() {
- return "Classpath for jar file "
+ this.zipFile.getName(); //$NON-NLS-1$
+ return "Classpath for jar file "
+ this.file.getPath(); //$NON-NLS-1$
}
public String normalizedPath(){
- String rawName = this.zipFile.getName();
+ String rawName = this.file.getPath();
return rawName.substring(0, rawName.lastIndexOf(’.’));
}
Figure 6. Incomplete refactoring induces a supplementary patch.
is a frequent source of omission errors. To investigate how
often supplementary bug fixes are induced by inconsistent
management of clones, we measure the content similarity
between a supplementary patch and its initial patch. The
patches are extracted by svndiff. The cloning relationship
between an initial patch and its supplementary patch is
identified by CCFinder [10] with a minimum token size of 5
tokens. We then exclude some patches if cloned code is less
than five lines. As Figure 7 shows, 11.92%, 25.35%, and
8.91% of supplementary patches include at least ve lines
similar to its initial patch.
Backporting patches are usually identical to original
patches, but are applied to different branches. We exclude
those from our analysis to accurately estimate the amount
of backporting patches out of all supplementary patches that
are similar to their initial patches.
Backported patches consist of 17.88%, 6.12%, and
12.34% in Eclipse JDT core, Eclipse SWT, and Mozilla re-
spectively. The majority of supplementary patches, 70.20%,
68.54%, and 78.75%, are not similar to initial patches.
The results are against the conventional wisdom that clone
management could significantly prevent or reduce omis-
sion errors. Our study result reveals that existing change
recommendation systems based on clone detection analysis
alone [3], [8], [9] are inadequate.
Predicting a supplementary fix location using code
clone analysis alone is insufficient.
D. Where Is The Location of Supplementary Bug Fixes in
Relation to Initial Fixes?
We investigate how the location of an initial patch is
related to the location of a supplementary patch in Java
subject programs to draw an insight into how to predict
supplementary change locations given an existing change
set. The files fixed in an initial patch could be modified
again during a supplementary fix, or new files could be fixed
instead. Out of 1048 and 1093 files fixed in supplementary
patches, 52.96% and 55.72% are applied to the same files
in Eclipse JDT core and Eclipse SWT respectively. To
determine changes made to similar line locations, we use
Yin et al.s heuristic [13], i.e., changes within 25 lines of
the original change. 48.09% and 41.54% are made within
25 lines of initial patch locations respectively.
To identify the structural dependence relationship between
the files of initial patches and the files of supplementary
patches, we use LSDiff to extract structural dependence
among source files. LSDiff infers systematic structural dif-
ferences as a logic rule [24], [25], and represents each pro-
gram version using a set of logic facts—e.g., calls (caller-
MethodName, calleeMethodName), access (fieldName,
accessorMethodName), etc. Using this technique, we ex-
tract structural facts for every released version of Eclipse
72
(11.92%)
108
(17.88%)
424
(70.20%)
Eclipse JDT core
145
(25.35%)
35
(6.12%)
392
(68.54%)
Eclipse SWT
620
(8.91%)
858
(12.34%)
5477
(78.75%)
Mozilla
A cloning relationship exists between an initial patch and its supplementary patch, but the supplementary patch is a backporting patch
A cloning relationship exists between an initial patch and its supplementary patch, which is not a backporting patch
No cloning relationship exists between an initial patch and its supplementary patch
Figure 7. The percentage of cloned patches and backported patches out of all supplementary patches
504
(48.09%)
51
(4.87%)
335
(31.97%)
158
(15.08%)
Eclipse JDT core
454
(41.54%)
174
(15.92%)
314
(28.73%)
151
(13.82%)
Eclipse SWT
Changes made within 25 lines of an initial patch
Changes made beyond 25 lines of an initial patch but on the
same files
Changes made to the files that directly depend on the initial patch
or the files on which an initial patch depends on
Changes that are not made to the same files of an initial patch
and that do not have a direct dependence relation with the initial
patch
Figure 8. 15.08% and 13.82% of files do not overlap with an initial patch location nor have direct dependencies on them.
JDT core and Eclipse SWT, and then we map the facts to
file-level dependence relationships. For example, the first
fix of the bug 111618 is on class ForeachStatement
and the third fix is on class AbstractVariable-
Declaration. Class ForeachStatement calls a
method named printAsExpression in Abstract-
VariableDeclaration, producing file-level depen-
dence relation between the initial patch and the supplemen-
tary patch. Out of all files of supplementary patches, 31.97%
and 28.73% are directly related to at least one file in the
initial patch as shown in Figure 8.
We also investigate indirect dependence relationships be-
tween the files of supplementary patches and files of initial
patches. A sibling relation means the two files extend from
the same ancestor type, and indirect call/access means
a file is indirectly called or accessed via a different file.
Overall, 15.08% and 13.82% of files in supplementary
patches do not overlap with nor have any direct relationship
to the files modified in the initial patch. This result indicates
that change recommendation systems that suggest the direct
dependence neighbors of an existing change set is not
enough for reducing omission errors.
About 15% of supplementary change locations are
beyond the scope of the direct neighbors of initial
patch locations.
V. DISCUSSION
Identification of Cloned Patches. In our study, a supple-
mentary fix revision is regarded as a cloned patch when it
has similar code chunk from an initial patch, longer than
five lines. However, it is possible that only few lines of a
supplementary patch is similar to the initial patch.
Figure 9 shows that the percentage of cloned patches
while varying the threshold. In Eclipse JDT core, 11.92% of
Table VII
THE STRUCTURAL DEPENDENCE RELATIONSHIPS BETWEEN AN INITIAL
PATCH AND ITS SUPPLEMENTARY PATCH AT THE FILE LEVEL
Relation type Eclipse JDT core Eclipse SWT
Direct
dependence
relations
Call 281 (57.00%) 295 (60.95%)
Access 244 (49.49%) 257 (53.10%)
Return 101 (20.49%) 53 (10.95%)
Fieldoftype 86 (17.44%) 61 (12.60%)
Extend 42 (8.52%) 26 (5.37%)
Implement 14 (2.84%) 0 (0.00%)
Indirect
dependence
relations
Sibling 40 (8.11%) 44 (9.09%)
Indirect call 85 (17.24%) 47 (9.71%)
Indirect access 46 (9.33%) 19 (3.93%)
Other 150 (30.43%) 134 (27.69%)
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
1 4 7 1013161922252831343740434649
Percentageofclonedpatches
ThenumberofsimilarlinesfoundbyCCFinder
EclipseJDTcore
EclipseSWT
Mozilla
Figure 9. The percentage of supplementary patches that are at least X
lines similar to their initial patches according to CCFinder, while varying
X.
supplementary patches are classified to cloned patches with
the threshold of ve lines. When the threshold of ten lines
are used, only 5.46% of supplementary patches are identical.
Relationships among Supplementary Patches. In our
study, we consider only the relationship between an initial
patch and its individual supplementary patches, as opposed
to the relationship among supplementary patches. For exam-
ple, in Eclipse JDT core, a bug 73104 has six fix attempts,
while the third patch reverts the second patch and the fourth
patch reverts the third one again. We also investigate the
time taken to fix a bug as the time from the initial patch to
the last supplementary patch, but we do not investigate on
the duration between consecutive patches.
The Studied Period of Bug Reports. The studied period
of bug reports are around two years in our study, which
are relatively short in comparison with the entire evolution
period of the subject systems. While the status of a bug
report in a bug database could be used to determine whether
each bug is resolved or not, even resolved bugs could be re-
opened for fix, as we’ve seen in our study. Thus, we limited
our focus to a part of the development period to ensure that
each bug is fully resolved. We acknowledge that the limited
scope of our study period could be a potential threats to
external validity, and we plan to extend the study duration
in the future.
New Tools for Reducing Incomplete Bug Fixes. Our
results show that a considerable amount of incomplete fixes
is caused by patches that do not cover all cases of condi-
tional statements correctly. An analysis tool that represents
differences in control flows of changed block might mitigate
such difficulties.
Our analysis also shows that an unfinished refactoring
often induces a supplementary fix. If a variable is refactored,
the methods accessing the variable should be refactored to-
gether, and some other methods calling the methods could be
refactored. Tools that can detect incomplete refactorings [26]
based on pre-defined or common refactoring error patterns
may prevent users from introducing omission errors.
VI. CONCLUSIONS
Many approaches have been proposed to prevent or reduce
omission errors or to recommend supplementary change
locations. These approaches make individual assumptions
on the characteristics of omission errors. To investigate why
omission errors occur in practice and how such errors can
be prevented, we studied the characteristics of incomplete
patches and supplementary patches.
Our study on three open source projects shows a con-
siderable portion of bugs requires supplementary bug fixes.
Incomplete patches tend to be larger and more scattered
than regular patches, and the common causes of omission
errors are diverse, including missed porting changes, in-
correct handling of conditional statements, or incomplete
refactorings, etc. In contrast to the conventional wisdom
that missed updates to code clones often cause omission
errors, only a small number of supplementary patches have
content similar to their initial patches. Recommending the
direct neighbors of initially fixed files in terms of structural
dependence or recommending clones of an existing change
set cannot predict all supplementary change locations. These
results call for new omission error reduction approaches that
complement existing change recommendation systems.
ACKNOWLEDGEMENTS
This research was sponsored by the Agency for De-
fense Development, Republic of Korea, under the grant
UD100031CD. This work was in part supported by National
Science Foundation under the grants CCF-1149391, CCF-
1043810, and CCF-1117902, and by a Microsoft SEIF
award.
REFERENCES
[1] Z. P. Fry and W. Weimer, “A human study of fault localization
accuracy, in Proceedings of the 2010 IEEE International
Conference on Software Maintenance. IEEE Computer
Society, 2010, pp. 1–10.
[2] A. E. Hassan and R. C. Holt, “Predicting change propagation
in software systems, in ICSM ’04: Proceedings of the 20th
IEEE International Conference on Software Maintenance.
IEEE Computer Society, 2004, pp. 284–293.
[3] T. T. Nguyen, H. A. Nguyen, N. H. Pham, J. Al-Kofahi,
and T. N. Nguyen, “Recurring bug fixes in object-oriented
programs, in ICSE ’10: Proceedings of the 32nd ACM/IEEE
International Conference on Software Engineering. ACM,
2010, pp. 315–324.
[4] M. P. Robillard, Automatic generation of suggestions for
program investigation, in ESEC/FSE-13: Proceedings of
the 10th European Software Engineering Conference held
jointly with 13th ACM SIGSOFT International Symposium
on Foundations of Software Engineering. ACM, 2005, pp.
11–20.
[5] T. Zimmermann, P. Weißgerber, S. Diehl, and A. Zeller,
“Mining version histories to guide software changes, IEEE
Transactions on Software Engineering, vol. 31, no. 6, pp.
429–445, 2005.
[6] A. T. T. Ying, G. C. Murphy, R. Ng, and M. Chu-Carroll,
“Predicting source code changes by mining change history.
IEEE Transactions on Software Engineering, vol. 30, no. 9,
pp. 574–586, 2004.
[7] T. Zimmermann, P. Weisgerber, S. Diehl, and A. Zeller,
“Mining version histories to guide software changes,” in ICSE
’04: Proceedings of the 26th International Conference on
Software Engineering. IEEE Computer Society, 2004, pp.
563–572.
[8] T. T. Nguyen, H. A. Nguyen, N. H. Pham, J. M. Al-Kofahi,
and T. N. Nguyen, “Clone-aware configuration management,
in ASE ’09: Proceedings of the 2009 IEEE/ACM International
Conference on Automated Software Engineering. IEEE
Computer Society, 2009, pp. 123–134.
[9] E. Duala-Ekoko and M. P. Robillard, “Tracking code clones
in evolving software, in ICSE ’07: Proceedings of the 29th
International Conference on Software Engineering. IEEE
Computer Society, 2007, pp. 158–167.
[10] T. Kamiya, S. Kusumoto, and K. Inoue, “CCFinder: A
multilinguistic token-based code clone detection system for
large scale source code. IEEE Transactions on Software
Engineering, vol. 28, no. 7, pp. 654–670, 2002.
[11] M. Kim, S. Sinha, C. Go andrg, H. Shah, M. Harrold,
and M. Nanda, Automated bug neighborhood analysis for
identifying incomplete bug fixes, in ICST ’10: Proceedings
of the third International Conference on Software Testing,
Verification and Validation. IEEE Computer Society, 2010,
pp. 383 –392.
[12] X. Zhang, S. Tallam, N. Gupta, and R. Gupta, “Towards
locating execution omission errors,” in PLDI ’07: Proceedings
of the 2007 ACM SIGPLAN conference on Programming
language design and implementation. ACM, 2007, pp. 415–
424.
[13] Z. Yin, D. Yuan, Y. Zhou, S. Pasupathy, and L. Bairavasun-
daram, “How do fixes become bugs?” in ESEC/FSE ’11:
Proceedings of the 19th ACM SIGSOFT symposium and
the 13th European conference on Foundations of software
engineering. ACM, 2011.
[14] Z. Gu, E. T. Barr, D. J. Hamilton, and Z. Su, “Has the bug
really been fixed?” in ICSE ’10: Proceedings of the 32nd
ACM/IEEE International Conference on Software Engineer-
ing - Volume 1. ACM, 2010, pp. 55–64.
[15] R. Purushothaman and D. E. Perry, “Toward understanding
the rhetoric of small source code changes,IEEE Transactions
on Software Engineering, vol. 31, no. 6, pp. 511–526, 2005.
[16] Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller,
“Documenting and automating collateral evolutions in linux
device drivers, in Eurosys ’08: Proceedings of the 3rd ACM
SIGOPS/EuroSys European Conference on Computer Systems
2008. ACM, 2008, pp. 247–260.
[17] J. Andersen and J. Lawall, “Generic patch inference, in
ASE ’08: Proceedings of the 23rd IEEE/ACM International
Conference on Automated Software Engineering, 2008. IEEE
Computer Society, Sept. 2008, pp. 337–346.
[18] X. Wang, D. Lo, J. Cheng, L. Zhang, H. Mei, and J. X. Yu,
“Matching dependence-related queries in the system depen-
dence graph, in Proceedings of the IEEE/ACM international
conference on Automated software engineering, ser. ASE ’10.
ACM, 2010, pp. 457–466.
[19] M. Kim, D. Cai, and S. Kim, An empirical investigation
into the role of refactorings during software evolution, in
ICSE’ 11: Proceedings of the 2011 ACM and IEEE 33rd
International Conference on Software Engineering. ACM,
2011.
[20] S. Kim, E. J. Whitehead, Jr., and Y. Zhang, “Classifying
software changes: Clean or buggy?” IEEE Transactions on
Software Engineering, vol. 34, no. 2, pp. 181–196, March
and April 2008.
[21] A. Mockus and L. G. Votta, “Identifying reasons for software
changes using historic databases, in ICSM ’00: Proceedings
of the International Conference on Software Maintenance.
IEEE Computer Society, 2000, p. 120.
[22] A. E. Hassan, “Predicting faults using the complexity of code
changes, in ICSE ’09: Proceedings of the 31st International
Conference on Software Engineering. IEEE Computer
Society, 2009, pp. 78–88.
[23] E. Gamma, R. Helm, R. Johnson, and J. M. Vlissides, De-
sign patterns: elements of reusable object-oriented software.
Addison-Wesley Professional, 1994.
[24] A. Loh and M. Kim, A program differencing tool to identify
systematic structural differences, in ICSE ’10: Proceedings
of the 2010 ACM and IEEE 32nd International Conference
on Software Engineering. ACM, 2010, pp. 263–266.
[25] M. Kim and D. Notkin, “Discovering and representing sys-
tematic code changes, in ICSE ’09: Proceedings of the 2009
IEEE 31st International Conference on Software Engineering.
IEEE Computer Society, 2009, pp. 309–319.
[26] C. G
¨
org and P. Weißgerber, “Error detection by refactoring
reconstruction, in MSR ’05: Proceedings of the 2005 inter-
national workshop on Mining software repositories. ACM
Press, 2005, pp. 1–5.
... We choose Apache open source projects as our subjects for they are active in the open-source community and most of them are also frequently investigated in bug prediction/detection research [3,4,8,13]. According to the ranking in OpenHub 10) , we finally selected 157 Apache open source projects, varying in sizes, domains, and functionalities. ...
... (2) |DepSet 1 | = 0 ∧ |DepSet 0 | > 0: this case can be mapped to Figure 2 Our toolkit iteratively and automatically runs each instance and gathers all the calculated differences 13 as the label results, which can further be integrated. ...
... To understand the scenarios of introducing dependency-level changes, we revisit the 140456 bugs in these 157 subjects from five aspects by considering dependency-level changes, including bug priority, bug fixing churn (lines of code), bug fixing time, bug reopening, and bug inducing, which are frequently investigated in bug characteristics research [13,14,19,20]. ...
Article
Full-text available
The complexity and diversity of bug fixes require developers to understand bug fixes from multiple perspectives in addition to fine-grained code changes. The dependencies among files in a software system are an important dimension to inform software quality. Recent studies have revealed that most bug-prone files are always architecturally connected with dependencies, and as one of the best practices in the industry, changes in dependencies should be avoided or carefully made during bug fixing. Hence, in this paper, we take the first attempt to understand bug fixes from the dependencies perspective, which can complement existing code change perspectives. Based on this new perspective, we conducted a systematic and comprehensive study on bug fixes collected from 157 Apache open source projects, involving 140456 bug reports and 182621 bug fixes in total. Our study results show that a relatively high proportion of bug fixes (30%) introduce dependency-level changes when fixing the corresponding 33% bugs. The bugs, whose fixes introduce dependency-level changes, have a strong correlation with high priority, large fixing churn, long fixing time, frequent bug reopening, and bug inducing. More importantly, patched files with dependency-level changes in their fixes, consume much more maintenance costs compared with those without these changes. We further summarized three representative patch patterns to explain the reasons for the increasing costs. Our study unveils useful findings based on qualitative and quantitative analysis and also provides new insights that might benefit existing bug prediction techniques. We release a large set of benchmarks and also implement a prototype tool to automatically detect dependency-level changes from bug fixes, which can warn developers and remind them to design a better fix.
... This can be due to omitted code changes and/or the need for fixing a mistake done in the first attempt to implement the change. Park et al. (2012) showed that 22% to 33% of bugs require more than one fix attempt (i.e., supplementary patches). Studying supplementary patches can be instrumental in designing recommender systems able to reduce omission errors by alerting software developers, as attempted in a subsequent work by Park et al. (2017), where the authors tried to predict additional change locations for real-world omission errors. ...
... Due to the limited empirical evidence about the nature of omitted changes, this is still an open challenge. Indeed, while the work by Park et al. (2012) investigates omitted changes, it explicitly focuses on supplementary patches for bug-fixing activities, ignoring other types of code changes (e.g., implementation of new features, refactoring). Thus, there is no study broadly investigating the types of changes that developers tend to omit during implementation activities. ...
... Hence, in most of the cases, the bug fixing process is not carried out by the same developers. Park et al. (2012) studied bugs whose initial patches were later considered incomplete and to which programmers applied supplementary patches. They examined three open source projects: Eclipse JDT core, Eclipse SWT, and Mozilla. ...
Article
Full-text available
Most changes during software maintenance and evolution are not atomic changes, but rather the result of several related changes affecting different parts of the code. It may happen that developers omit needed changes, thus leaving a task partially unfinished, introducing technical debt or injecting bugs. We present a study investigating “ quick remedy commits ” performed by developers to implement changes omitted in previous commits. With quick remedy commits we refer to commits that (i) quickly follow a commit performed by the same developer, and (ii) aim at remedying issues introduced as the result of code changes omitted in the previous commit (e.g., fix references to code components that have been broken as a consequence of a rename refactoring) or simply improve the previously committed change (e.g., improve the name of a newly introduced variable). Through a manual analysis of 500 quick remedy commits, we define a taxonomy categorizing the types of changes that developers tend to omit. The taxonomy can (i) guide the development of tools aimed at detecting omitted changes and (ii) help researchers in identifying corner cases that must be properly handled. For example, one of the categories in our taxonomy groups the reverted commits , meaning changes that are undone in a subsequent commit. We show that not accounting for such commits when mining software repositories can undermine one’s findings. In particular, our results show that considering completely reverted commits when mining software repositories accounts, on average, for 0.07 and 0.27 noisy data points when dealing with two typical MSR data collection tasks (i.e., bug-fixing commits identification and refactoring operations mining, respectively).
... To make developers more productive in the process, different techniques exist: Software testing [10] and formal verification [4] figure out whether bugs exist somewhere, and automated fault localization [16] can propose code locations that may be buggy. But even fixing a known bug seems to pose challenges: Multiple studies [1,7,11,12,13,14,17] on selected software projects showed a large number of fixes in these projects did not actually fix the bug. (Such fixes are also known as partial or incomplete patches.) ...
... Multiple benchmark sets exist for automated program repair [3,5,8] and the existence of partial fixes has already been studied [7,11,12,14,17]. Unfortunately, a benchmark set for evaluating automated program repair with partial fixes does not exist. ...
Preprint
Full-text available
Software bugs significantly contribute to software cost and increase the risk of system malfunctioning. In recent years, many automated program-repair approaches have been proposed to automatically fix undesired program behavior. Despite of their great success, specific problems such as fixing bugs with partial fixes still remain unresolved. A partial fix to a known software issue is a programmer's failed attempt to fix the issue the first time. Even though it fails, this fix attempt still conveys important information such as the suspicious software region and the bug type. In this work we do not propose an approach for program repair with partial fixes, but instead answer a preliminary question: Do partial fixes occur often enough, in general, to be relevant for the research area of automated program repair? We crawled 1500 open-source C repositories on GitHub for partial fixes. The result is a benchmark set of 2204 benchmark tasks for automated program repair based on partial fixes. The benchmark set is available open source and open to further contributions and improvement.
... The prior work [10, 11,12,13] shows that developers may commit errors 80 of omission (i.e., forgetting to apply edits completely) when they have to edit multiple program locations simultaneously in one maintenance task (i.e., bug fixing, code improvement, or feature addition). For instance, Fry et al. [10] reported that developers are over five times more precise at locating errors of commission than errors of omission. ...
... For instance, Fry et al. [10] reported that developers are over five times more precise at locating errors of commission than errors of omission. Yin et al. [12] and Park et al. [13] 85 separately showed that developers introduced new bugs when applying patches to fix existing bugs. In particular, Park et al. inspected the supplementary bug fixes following the initial bug-fixing trials, and summarized nine major reasons to explain why the initial fixes were incorrect. ...
Article
Full-text available
JavaScript (JS) is one of the most popular programming languages due to its flexibility and versatility, but maintaining JS code is tedious and error-prone. In our research, we conducted an empirical study to characterize the relationship between co-changed software entities (e.g., functions and variables), and built a machine learning (ML)-based approach to recommend additional entity to edit given developers’ code changes. Specifically, we first crawled 14,747 commits in 10 open-source projects; for each commit, we created at least one change dependency graph (CDG) to model the referencer-referencee relationship between co-changed entities. Next, we extracted the common subgraphs between CDGs to locate recurring co-change patterns between entities. Finally, based on those patterns, we extracted code features from co-changed entities and trained an ML model that recommends entities-to-change given a program commit. According to our empirical investigation, (1) three recurring patterns commonly exist in all projects; (2) 80%–90% of co-changed function pairs either invoke the same function(s), access the same variable(s), or contain similar statement(s); (3) our ML-based approach CoRec recommended entity changes with high accuracy (73%–78%). CoRec complements prior work because it suggests changes based on program syntax, textual similarity, as well as software history; it achieved higher accuracy than two existing tools in our evaluation.
... Developers often spend a substantial amount of time diagnosing a configurable software system to localize and fix a performance bug, or to determine that the system was misconfigured [8,11,26,30,32,33,55,58,59,86]. This struggle is quite common when maintaining configurable software systems. ...
Preprint
Full-text available
Determining whether a configurable software system has a performance bug or it was misconfigured is often challenging. While there are numerous debugging techniques that can support developers in this task, there is limited empirical evidence of how useful the techniques are to address the actual needs that developers have when debugging the performance of configurable software systems; most techniques are often evaluated in terms of technical accuracy instead of their usability. In this paper, we take a human-centered approach to identify, design, implement, and evaluate a solution to support developers in the process of debugging the performance of configurable software systems. We first conduct an exploratory study with 19 developers to identify the information needs that developers have during this process. Subsequently, we design and implement a tailored tool, adapting techniques from prior work, to support those needs. Two user studies, with a total of 20 developers, validate and confirm that the information that we provide helps developers debug the performance of configurable software systems.
Conference Paper
Full-text available
Localizing and repairing defects are critical software engineering activities. Not all programs and not all bugs are equally easy to debug, however. We present formal models, backed by a human study involving 65 participants (from both academia and industry) and 1830 total judgments, relating various software- and defect-related features to human accuracy at locating errors. Our study involves example code from Java textbooks, helping us to control for both readability and complexity. We find that certain types of defects are much harder for humans to locate accurately. For example, humans are over five times more accurate at locating “extra statements” than “missing statements” based on experimental observation. We also find that, independent of the type of defect involved, certain code contexts are harder to debug than others. For example, humans are over three times more accurate at finding defects in code that provides an array abstraction than in code that provides a tree abstraction. We identify and analyze code features that are predictive of human fault localization accuracy. Finally, we present a formal model of debugging accuracy based on those source code features that have a statistically significant correlation with human performance.
Conference Paper
Full-text available
A key issue in maintaining Linux device drivers is the need to update drivers in response to evolutions in Linux internal libraries. Currently, there is little tool support for performing and documenting such changes. In this paper we present a tool, spdiff, that identifies common changes made in a set of pairs of files and their updated versions, and extracts a generic patch performing those changes. Library developers can use our tool to extract a generic patch based on the result of manually updating a few typical driver files, and then apply this generic patch to other drivers. Driver developers can use it to extract an abstract representation of the set of changes that others have made. Our experiments on recent changes in Linux show that the inferred generic patches are more concise than the corresponding patches found in commits to the Linux source tree while being safe with respect to the changes performed in the provided pairs of driver files.
Conference Paper
Full-text available
Abstract Predicting the incidence of faults in code has been com- monly associated with measuring complexity. In this paper, we propose complexity metrics that are based on the code change process instead of on the code. We conjecture that a complex code change process negatively affects its prod- uct, i.e., the software system. We validate our hypothesis empirically through a case study using data derived from the change history for six large open source projects. Our case study shows that our change complexity metrics are better predictors of fault potential in comparison,to other well-known historical predictors of faults, i.e., prior modi- fications and prior faults.
Conference Paper
In many cases it is not sufficient to perform a refactoring only at one location of a software project. For example, refactorings may have to be performed consistently to several classes in the inheritance hierarchy, e.g. subclasses or implementing classes, to preserve equal behavior.In this paper we show how to detect incomplete refactorings - which can cause long standing bugs because some of them do not cause compiler errors - by analyzing software archives. To this end we reconstruct the class inheritance hierarchies, as well as refactorings on the level of methods. Then, we relate these refactorings to the corresponding hierarchy in order to find missing refactorings and thus, errors and inconsistencies that have been introduced in a software project at some point of the history.Finally. we demonstrate our approach by case studies on two open source projects.
Conference Paper
Software bugs affect system reliability. When a bug is exposed in the field, developers need to fix them. Unfortunately, the bug-fixing process can also introduce errors, which leads to buggy patches that further aggravate the damage to end users and erode software vendors' reputation. This paper presents a comprehensive characteristic study on incorrect bug-fixes from large operating system code bases including Linux, OpenSolaris, FreeBSD and also a mature commercial OS developed and evolved over the last 12 years, investigating not only themistake patterns during bug-fixing but also the possible human reasons in the development process when these incorrect bug-fixes were introduced. Our major findings include: (1) at least 14.8%--24.4% of sampled fixes for post-release bugs in these large OSes are incorrect and have made impacts to end users. (2) Among several common bug types, concurrency bugs are the most difficult to fix correctly: 39% of concurrency bug fixes are incorrect. (3) Developers and reviewers for incorrect fixes usually do not have enough knowledge about the involved code. For example, 27% of the incorrect fixes are made by developers who have never touched the source code files associated with the fix. Our results provide useful guidelines to design new tools and also to improve the development process. Based on our findings, the commercial software vendor whose OS code we evaluated is building a tool to improve the bug fixing and code reviewing process.
Conference Paper
Before performing a modification task, a developer usually has to investigate the source code of a system to understand how to carry out the task. Discovering the code relevant to a change task is costly because it is an inherently human activity whose success depends on a large number of unpredictable factors, such as intuition and luck. Although studies have shown that effective developers tend to explore a program by following structural dependencies, no methodology is available to guide their navigation through the typically hundreds of dependency paths found in a non-trivial program. In this paper, we propose a technique to automatically propose and rank program elements that are potentially interesting to a developer investigating source code. Our technique is based on an analysis of the topology of structural dependencies in a program. It takes as input a set of program elements of interest to a developer and produces a fuzzy set describing other elements of potential interest. Empirical evaluation of our technique indicates that it can help developers quickly select program elements worthy of investigation while avoiding less interesting ones.
Conference Paper
Previous research confirms the existence of recurring bug fixes in software systems. Analyzing such fixes manually, we found that a large percentage of them occurs in code peers, the classes/methods having the similar roles in the systems, such as providing similar functions and/or participating in similar object interactions. Based on graph-based representation of object usages, we have developed several techniques to identify code peers, recognize recurring bug fixes, and recommend changes for code units from the bug fixes of their peers. The empirical evaluation on several open-source projects shows that our prototype, FixWizard, is able to identify recurring bug fixes and provide fixing recommendations with acceptable accuracy.
Conference Paper
It is widely believed that refactoring improves software quality and programmer productivity by making it easier to maintain and understand software systems. However, the role of refactorings has not been systematically investigated using fine-grained evolution history. We quantitatively and qualitatively studied API-level refactorings and bug fixes in three large open source projects, totaling 26523 revisions of evolution. The study found several surprising results: One, there is an increase in the number of bug fixes after API-level refactorings. Two, the time taken to fix bugs is shorter after API-level refactorings than before. Three, a large number of refactoring revisions include bug fixes at the same time or are related to later bug fix revisions. Four, API-level refactorings occur more frequently before than after major software releases. These results call for re-thinking refactoring’s true benefits. Furthermore, frequent floss refactoring mistakes observed in this study call for new software engineering tools to support safe application of refactoring and behavior modifying edits together.
Conference Paper
Program differencing tools such as GNU diff identify individual differences but do not determine how those differences are related to each other. For example, an extract superclass refactoring on several subclasses will be represented by diff as a scattered collection of line additions and deletions which must be manually pieced together. In our previous work, we developed LSdiff, a novel program differencing technique that automatically identifies systematic structural differences as logic rules. This paper presents an LSdiff Eclipse plug-in that provides a summary of systematic structural differences along with textual differences within an Eclipse integrated development environment. This plugin provides several additional features to allow developers to interpret LSdiff rules easily, to select the abstraction level of program differencing analysis, and to reduce its running time through incremental program analysis.