Content uploaded by Auri Marcelo Rizzo Vincenzi
Author content
All content in this area was uploaded by Auri Marcelo Rizzo Vincenzi on May 11, 2023
Content may be subject to copyright.
MutantOperators for Testing ConcurrentJavaPrograms
M´arcio Delamaro
Universidade Estadual de Maring´a(UEM)
delamaro@din.uem.br
Mauro Pezz`e
Universit`adegli Studi di Milano
pezze@disco.unimib.it
Auri M. R. Vincenzi
Jos´eCarlos Maldonado
Universidade de S˜ao Paulo (USP)
{jcmaldon,auri}@icmc.sc.usp.br
Abstract
Mutation testing is afault-based testing technique that has been widely studied
in the last decades. One reason for the interest of the scientific communityin mu-
tation testing is its flexibility.It can beapplied to programs at unit and integration
testing levels, as well as to software specifications written in avarietyof different
languages. Afundamental issue to makemutation testing work for agiven language
or environmentis the set of mutantoperators used to create the mutants. In this
paper aset omutantoperator is proposed for the Javaprogramming language, and
more specifically,aiming at exercising aspects of concurrency and synchronization
of that language.
1Introduction
Testing is acrucial activityin the software lifecycle. It is expensiveand time con-
suming. For this reason mucheffort has been spenton developing techniques and
tools to support the testing activity.An importantresult of researchis the defi-
nition techniques and criteria to drivethe generation of test sets that can suitably
exercise aprogram.
Mutation testing is afault based test technique. It uses aset of “rules” called
mutantoperators to create programs slightly differentfrom the program under
test. These programs are called mutants.The goal of mutation testing is the
generation of atest set that distinguishes the behavior of the mutants from the
original program. The ratio of distinguished mutants (also called dead or killed
mutants), over the total number of mutants, measures the adequacy of the test set.
According to the coupling effect hypothesis[14], test cases that distinguish
simple faults injected in the original program to create the mutants should also
beable to reveal faults that can beobtained as acomposition of simple faults.
XV Simpósio Brasileiro de Engenharia de Software
- 272 -
Thus, mutantoperators can beseen as representing common faults usually found
in software.
Mutantoperators depend on the language in whichthe artifact to betested
is described. So far mutation testing has been applied to programs written in
several programming languages, likeFortran [5], C[3] and Java[11]. and to formal
specifications written using Finite State Machines [7], Petri Nets [8], Statecharts [6]
and Estelle [20, 18]. This flexibilityderives from the fact that mutation testing
requires only an executable model that transforms inputs into observable outputs
that can becompare against the results produced bythe mutants.
Mutantoperators are designed referring to the experience of using the target
language and of the most common faults. In the past mutantoperators were de-
signed based experts’ knowledge. Recently,Kim et al. [10] haveproposed the use
of atechnique named “Hazard and OperabilityStudies” (HAZOP) to systemati-
cally derivemutantoperators and they applied it to the Javalanguage. Although
the resulting operators do not significantly differ from past works, the proposed
methodology is an importantstep towards amore rigorous discipline in the creation
of mutantoperators. The technique is based on twomain concepts. It first iden-
tifies in the grammar of the taget language (Javain this case) the points subject
to mutation and then defines the set of mutations these points based on suuitable
“Guide Words”.
In general, acomprehensiveset of mutantoperators, that well cover the struc-
tures of agiven language tends to produce toomanymutants even for small pro-
grams. This problem has been tackled byselecting subsets of mutants according
to differentstrategies [21, 16, 13] likerandomly selecting apercentage of the gen-
erated mutants or choosing asubset of the mutantoperators based on some known
characteristic likecost or tendency to generate equivalentmutants.
The test of concurrentprograms is complicated bythe nondeterminism of their
execution, that in the case of mutation testing, makes it difficult to analyze the
behavior of the mutants. Since aprogram in this context can presentdifferent
correct results, the fact that the behavior of amutantdiffers from the result of the
original program for atest case tdoes not mean that thas revealed the fault in the
mutant. The discrepancy mayresult from the non determinism of the program and
not from the fault seeded in the mutant.
Twodifferentapproaches to apply mutation testing to concurrentprograms have
been proposed in the literature [17, 19]. Both haveADAas their target language and
are presented with more details in the next section. Unfortunately the approaches
do not extend straightforwardly to other languages, suchas Java, that are not
based on the Ada rendezvous model. Apilot study that used apre-existentmutant
operator set confirmed the need of mutantoperators specifically designed for the
Javamodel of concurrency and synchronization [2].
This paper describes part of the work that aims at applying mutation testing to
Javaconcurrentprograms. Herein wedescribethe design of mutantoperators. In
Section 2weoverview previous work on mutation testing for concurrentprograms,
and in particular the work of Silva-Barradas [19] that is closely related to our goals.
In Section 3webriefly explains the mechanisms used for synchronization and con-
currency in Java. In Section 4wepresentthe set of mutantoperators designed
to exercise those mechanisms. The conclusions and ongoing works related to this
paper are presented in Section 5.
XV Simpósio Brasileiro de Engenharia de Software
- 273 -
2Mutation Testing for ConcurrentPrograms
When testing asequential program, one can rely on the fact that there exists only
one correct output for agiven test data. The statements are executed in adeter-
mined order that does not change from one execution to another, if the input is the
same. This is not true for concurrentprograms. The parallel or concurrentexecu-
tion of several (deterministic) processes mayresult in differentoutputs, depending
on the order of execution of the differentprocesses.
This is particularly troublesome for mutation testing. When amutantMis
executed with atest case t,it should bekilled if its behaviour is incorrect, according
to agiven specification. In the case of asequential program this is equivalentto say
that the results of the execution of Mwith tdiffers from the results of the execution
of the original program Pwith t.In the case of concurrentprograms weshould
compare the set of all possible results of Mwith tagainst the set of all possible
results of Pwith tand then consider Mdistinguished if they are not the same.
Unfortunately,in general it is not possible to knowthe set of all possible results
of aconcurrentprogram for agiven input. Offut et al. [17] proposed atechnique
that calculates an approximated set Ω, defined as asubset of all possible results
of P(t). This set is calculated byexecuting P(t)several times. The mutantMis
distinguished if it produces at least one result M(t)/∈Ω.
Another approach, called “Behavior Mutation Analysis” (BMA), is defined by
Silva-Barradas [19]. Asuitable instrumentation of an ADAprogram Pcan beused
to register the sequence of execution of its tasks. The recorded sequence can beused
to reproduce the same behavior in asubsequentexecution of P(t). Incorporating
this synchronization sequence as part of atest case, it is expected that M(t,s)=
P(t,s), i.e., the result of the execution of the mutantMwith treproducing the same
synchronization sequence should bethe same of the original program. The BMA
technique considers the mutantdistinguished if one of the twoconditions is verified:
1) M(t,s)=P(t,s); or 2) Mcannot reproduce the synchronization sequence s.The
second case mayoccur, for example, if the mutation eliminates astatementthat
when executed adds an eventto the synchronization sequence, e.g., a entry call. In
this case, BMA considers that the mutanthas an incorrect behavior (deadlockor
abnormal termination) and thus should bedistinguished.
In apilot study [2] wetried this approachusing existing operators [4] adapted to
Javaand wecould derivesome importantobservations that are guiding the current
developmentof the technique for the Javalanguage.
•Amutantcan bedistinguished byeither producing adifferentoutput or be-
cause it cannot reproduce the same synchronization sequence of the original
program. In general amutantthat can bedistinguished byone reason can-
not bedistinguished bythe other. In afew cases though it is possible to
find mutants that can bedistinguished from the original program both be-
cause producing adifferentoutput for atest case t1and because not able to
reproduce the required synchronization sequence for adifferenttest case t2.
Although in general mutants should bekilled if distinguished in either ways,
for suchmutants it would beuseful to require both cases for killing them, since
the differenttest cases can reveal differentfaults;
•The number of equivalentmutants, i.e., mutants that cannot bedistinguished
byeither reasons, is very low. In the pilot study only 2out of the 144 mutants
XV Simpósio Brasileiro de Engenharia de Software
- 274 -
public class myclass {
public synchronized void myMethod()
{
doSomething();
}
public void myOtherMethod()
{
doSomeOtherThing();
}
}
Figure 1: Example of synchronized method
created were equivalents;
•It is harder to analyze mutants when concurrency is involved, mainly when it
is necessary to induce certain specific synchronization sequence to distinguish
them. Auseful help in this matter would bethe construction of atool that
could generate differentsynchronization sequences to test the program. We
already knowthat atool is necessary to record the synchronization sequences
of the test cases so they can bereproduced in the mutant. In the ongoing
work of developing suchtool weare studying ways to create valid variants of
an original synchronization sequence. Suchvariants can beuseful for testing
the program and killing the mutants.
•The set of mutantoperators, based on the Interface Mutation operators [4],
used in the pilot study is not sufficient. Although wecould not formally eval-
uate the qualityof the test set obtained –because no other criterion exists
(in our best knowledge) against whichwecould evaluate it –wecould intu-
itively note that some characteristics (either functional and structural) related
to concurrency where not exercised and an existing bug could not berevealed.
In the next section wesummarize the features of the Javalanguage related to
concurrency and synchronization and that conduced to the mutantoperator set
described in Section 4.
3Overview of JavaConcurrency Mechanisms
The basic mechanism for thread synchronization in Javaare synchronized methods
and blocks. Every object in Javahas amonitor associated to it. This monitor is
used to guarantee that only one thread at time has access to the object. Figure 1
shows an example of synchronized method. In aprogram with twoor more threads
sharing an object Xof typemyClass,only one at atime can enter myMethod using
that object. The access to myOtherMethod does not havesuchrestriction. So a
thread T1can execute, for instance, myMethod on object Xconcurrently with thread
T2,executing myOtherThread on the same object X.
If agiven thread T1is executing asynchronized methodon an object Xand
another thread T2tries to enter asynchronized methodon the same object the
XV Simpósio Brasileiro de Engenharia de Software
- 275 -
free monitor free monitor 2
lock monitor lock monitor 2
want lock want lock 2
Monitor
Accessing obj Accessing obj 2
Contation Contation 2
Exc Normal Exc Normal 2
Figure 2: Basic model of 2threads sharing an object
system blocks T2until T1terminates executing the synchronized method. Then T2
can execute, or more precisely,it can compete to get access to the object, since other
threads mayalso havebeen trying to access the same object within asynchronized
method.
When athread T1gains access to asynchronized methodof an object Xwe
saythat T1owns or has locked X’s monitor. Astatic methodcan besynchronized.
In this case entering the methodlocks the monitor associated to the Class object
associated to the class where the methodis defined.
Figure 2shows asimple Petri Net model of twothreads that share an object X.
The places named Exec Normal representthe states of eachthread when they are
executing non synchronized code w.r.t. the object of interest. Athread maywantto
execute asynchronized methodon Xand then it reaches a contention state, where
it will lockthe monitor, if it is available. On succeeding, the token is removed from
the monitor place and is holden until the thread finishes the synchronized method
and releases the monitor.
Synchronized blocks are similar to synchronized methods but protected code is
restricted only to apiece of amethodand the object on whichthe lockis exe-
cuted is explicitly declared. For example, in the code in Figure 3, the thread must
obtain the monitor of myObject before executing doSomeOtherThing,in method
myOtherMethod.
Every object created in the JVM has associated await set whichallows athread
in possess of amonitor to temporarily release that monitor until an eventoccurs.
Class Object defines await methodthat releases the monitor of the object used in
XV Simpósio Brasileiro de Engenharia de Software
- 276 -
public class myclass {
myOtherClass myObject = new myOtherClass();
public synchronized void myMethod()
{
doSomething();
}
public void myOtherMethod()
{
synchronized (myObject)
{
doSomeOtherThing();
}
}
}
Figure 3: Example of synchronized block
the call and insert the currentthread in the object wait set, temporarily blocking
its execution. The definition of methodwait explicitly states that the thread that
calls the wait must beholding the monitor of the object, otherwise an exception is
thrown. Since every class in Javainherits from class Object,every object created
in Javahas amethodwait (declared final in the class Object).
Class Object also declares amethodnotify whichwakes up athread waiting
in await set. When this methodis called on an object Xone of the threads is
randomly removed from the wait set, becoming ready to execute. This does not
mean that it will beexecuted immediately.First because the thread that calls
notify must bein possess of the object monitor, thus the thread removed from
the wait set can execute only after the currentthread releases the lock. Moreover,
even after the currentthread has released the lock, there is no guarantee that that
specific thread will bethe one that acquires the monitor because other threads may
bein contention for the same monitor.
The thread removed from the wait set will resume from the pointimmediately
after the call to wait and its state returns to the same it had at that time, i.e.,
in possess of the monitor and with the same lockcountit had on that monitor.
For example, if it had locked the monitor twice, it reacquires the monitor with lock
countequals two. Figure 4shows the example of twothreads that share an object,
including the call of methodwait.Note that the model becomes significantly more
complex than that shown in Figure 2.
The wait methodhas avariantthat accepts as argumentatimeout value. The
semantics of this call is similar to the one described above, i.e., the thread is inserted
in the wait set until anotify removes it. In addition, if it is not removed byanotify
within the specified timeout, it is removed “by itself”. The notify methodhas a
variantnotifyAll that, when called on object X,removes all the threads from the
wait set of object X.
Other methods in the API complete the resources of the language for concur-
rency.Among them, the methods in the class Thread,responsible for the creation
and managementof new threads.
XV Simpósio Brasileiro de Engenharia de Software
- 277 -
get notify get notify 2
wait wait 2
free monitor free monitor 2
lock monitor lock monitor 2
want lock want lock 2
wait list
wait list 2
Monitor
Accessing obj Accessing obj 2
Contation Contation 2
Exc Normal Exc Normal 2
Figure 4: Model of twothreads sharing an object, including wait sets
4MutantOperators for Java
In this section wepresentaset of mutantoperators designed to exercise the con-
currency and synchronization aspects of Javaprograms. First weidentify the main
structures related to concurrency and then wederivemutantoperators to exercise
suchstructures. One essential aspect taken in consideration is that the practical ap-
plicability on mutation testing heavily depends on the number of mutants generated
bythe approach.
Based on the overview presented in Section 3weidentified four groups of struc-
tures of Javarelated to concurrency:
•Monitor lockcode;
•Methods related to wait set manipulation that are defined in the Javacore
API;
•Use of synchronized methods;
•Use of other methods related to synchronization and concurrency.
Eachof these classes are analyzed and mutation operator to exercise them are
proposed in the following sections.
XV Simpósio Brasileiro de Engenharia de Software
- 278 -
synchronized public void append (Element x)
{
while (cont >= buffer.length)
wait();
buffer[nextSpot] = x;
nextSpot = (nextSpot + 1) % buffer.length;
// nextSpot = (nextSpot * 1) % buffer.length; <<== mutation
cont++;
notify();
}
Figure 5: Asynchronized methodand apossible mutant
4.1 Monitor locks
The class of monitor locks include synchronized methods and statements. Amethod
is declared synchronized adding the keyword synchronized in frontof its declara-
tion, for example
synchronized public foo(int k, String s)
{
...
}
The first mutantoperator identified for this construc is the deletion of the key-
word synchronized,turning the methodto “normal”, regardless of the monitor
acquisition. This will require test cases that exercise the concurrentuse of each
synchronized method.
The objectiveof amutation is to create aprogram with behavior that slightly
differs fro the behavior of the original program. In the particular case of synchro-
nized methods, wewantamethodthat when called will produce a differentbehavior
related to the access to ashared object. This could beachieved bymutating any
statementin the body of the method. Wecould use mutantoperators similar to
those previously defined for unit testing [1, 12, 11] to obtain the discrepantbehavior.
For example, the code in Figure 5shows asimple methodthat inserts an element
in acircular shared buffer, used in aproducer-consumer system, and apossible
mutant.
If on one hand this mutantmayhelp to exercise importantfeatures in the pro-
gram, forcing for example atest case that uses abuffer larger than one element,
on the other hand all suchmutants would makethe test impractical. First because
the number of mutants would growto an undesirable level. Second, because these
“fine grained” mutants would change the state of the program in suchawaythat
would makevery hard for the tester to analyze them. In apilot study [2], the
use of Interface Mutation operators [4] confirmed these twoconjectures. To avoid
these problems wedefine only one typeof mutation to beapplied in the body of a
synchronized method, i.e., statementdeletion, as defined in [1].
This operator removes one statementat eachtime. In this waywereduce the
number of mutants and create mutants that are easier to beanalyzed because they
XV Simpósio Brasileiro de Engenharia de Software
- 279 -
representmore significantand understandable changes in the code. Using the def-
inition of Offut and Hayes [15], wehavemutants with alarger syntactic difference
from to the original program and as side effect it is expected also alarger semantic
difference.
Asynchronized statementis defined in [9] as
SynchronizedStatement: synchronized (Expression )Block
The twomutantoperators defined for synchronized methods can beapplied. The
first removes the declaration of the blockas synchronized and the second deletes
astatementinside the block. Wecan also apply operator “movebrace up/down”
as defined byAgrawal et al. [1] to the synchronized block. This operator moves
the closing brace one statement“up” moving the last statementout of the blockor
one statement“down” including in the blockthe first statementdeclared outside.
Wecan apply asimilar operator to the beginning of the block, excluding the first
statementor inserting the immediately previous one. These operators generate only
few mutants (at most four per synchronized block) and are amore subtle version
of the statementdeletion operator. Test cases that distinguish suchmutants might
beuseful to verify the limits that asynchronized statementmust have.
The semantic of asynchronized statementindicates that its synchronization
expression should evaluate to an object, i.e., an instance of java.lang.Object or
one of its subclasses. Toguarantee that the programmer used the correct expression,
i.e., that the synchronization is being done in the rightobject wecan replace the
synchronization expression byother objects accessible in that scope. So wedefine
four mutantoperators:
•replace the synchronization expression with alocal variable;
•replace the synchronization expression with aformal parameter;
•replace the synchronization expression with astatic field; and
•replace the synchronization expression with an instance field;
4.2 Wait sets
In this section wepresentthe operators designed to exercise another main feature
of the Javasynchronization mechanism: the wait sets. Basically,the wait sets are
managed bytwotypes of methods: 1) those that ask the system to blockathread
inserting it in await set (wait and timed wait methods); and 2) those that ask the
system to removethreads from await set (notify and notifyAll).
The mutantoperators designed to deal with the use of the wait methods are:
•Delete acall to wait.In this case the thread that should beblocked continues
executing. Test cases that kill this kind of mutants showthe relevance of each
wait call. This maybeparticularly importantfor timed wait that mayrequire
very specific test cases to beeffectively used.
•Replace a timed wait with anon-timed.
•Replace anon-timed wait with atimed.
•Incrementand decrementtime in a timed wait.These last three operators
exercise timing aspects of aprogram.
XV Simpósio Brasileiro de Engenharia de Software
- 280 -
For exercising the use of notify methods wedefine:
•Delete acall to notify or notifyAll.This operator changes the set of threads
available for execution at agiven pointof the execution. Test cases must be
provided in order to showthat eachof suchcalls is really necessary.
•Replace acall to notify with acall to notifyAll.
•Replace acall to notifyAll with acall to notify.These twooperators might
beuseful for testing synchronization aspects because test cases that distinguish
them mayrequire the execution of very specific sequences of wait and notify
statements, otherwise difficult to exercise.
4.3 Synchronized methodcalls
The operators in this class aim at exercising the interactions with methods declared
as synchronized as they are amain feature of the Javalanguage, for what concern
synchronization. Note that these operators differ from those defined in Section 4.1
because those are related to the declaration of synchronized methods and the ones
defined here are related to the use of suchmethods, i.e., the invocation of such
methods.
The typeof interactions wewantto exercise here are similar to those exercised
bythe Interface Mutation approach[4]. Interface Mutation defines aset of mutant
operators that focus on the interaction between twounits, e.g., twofunctions in the
Clanguage. Here wewantto specifically exercise the use of synchronized methods,
changing the points where they are invoked. Based on the set of Interface Mutation
operators wedefine the following operators:
•Replace arguments with constants. Eachargumentis replaced with constants
defined according to the typeof the argument. For example int arguments
can bereplaced with {0, 1, -1, Integer.MAX VALUE, Integer.MIN VALUE}
where MAX VALUE and MIN VALUE are respectively the largest positive
integer and the smallest negativeinteger as defined bythe JavaAPI. The sets
of required constants for eachtypein the language is shown in Table 1;
•Delete the call. If the methodreturns avalue that is used in an expression
then the call is replaced with constants of the typeof the returned value;
•Switcharguments of compatible types;
•Change methodsignature. Change the methodcall byremoving one of the
arguments or adding anew argument, changing the call to matchthe decla-
ration of another methodwith the same name but with differentsignature, if
suchmethodexists;
•Insert arithmetic negation before an argument. Invert the sign of an argument
byinserting an arithmetic negation, if allowed bythe typeof the argument.
If the argumentis an expression, only the whole expression is negated, not
subexpressions;
•Insert logical negation before an argument;
•Insert bit negation before an argument;
•Incrementand decrementan argument;
XV Simpósio Brasileiro de Engenharia de Software
- 281 -
•Change target object. Use another type-compatible object to makethe call.
Wecan subdivide this mutation depending on whichobject is used to replace
the original one. So wecan have: 1) replace with local variable; 2) replace
with formal parameter; 3) replace with static field; 4) replace with instance
field.
Table 1: Set of constants for operator “Replace arguments with constants”
TypeConstants
int0, 1, -1, Integer.MIN VALUE, Integer.MAX VALUE
short 0, 1, -1, Short.MIN VALUE, Short.MAX VALUE
byte 0, 1, -1, Byte.MIN VALUE, Byte.MAX VALUE
long 0, 1, -1, Long.MIN VALUE, Long.MAX VALUE
char 1, Character.MIN VALUE, Character.MAX VALUE
boolean true, false
float 0.0, 1.0, -1.0, Float.MIN VALUE, Float.MAX VALUE
double 0.0, 1.0, -1.0, Double.MIN VALUE, Double.MAX VALUE
String ””, null
other objects null
4.4 Other synchronization methods
Other methods that are not part of the “core” mechanisms for synchronization in
Javahavebeen defined in the JavaAPI and are importantfor the implementation of
concurrentprograms. Tothese methods wecan apply the same mutations defined
in the last section, changing their behavior bymutating the calls to them.
The methods weidentify in this class of mutations are: Thread.interrupt,
Thread.join,Thread.sleep,Thread.start and Thread.yeld.
Note that some operators, when applied to specific structures in aprogram can
generate the same mutation, or more precisely,mutants with the same behavior.
This is the case for example of deleting astatementin asynchronized blockand
deleting await call. If the first is applied, the second does not haveto beapplied.
This is not uncommon in the definition of mutantoperators and allows one to use
an incremental approachto choose the mutantoperators.
Muchof the mutantoperator set can beparameterized according to the the
implementor’s or even the tester’s will. For example: the value to add/subtract on
timed wait;the set of constants to use in the replacements described in Section 4.3;
the methods subject to mutation described in Section 4.4.
Table 2summarizes the set of mutantoperators.
5Conclusion and Future Work
This paper discussed the use of mutation testing for concurrentJavaprograms.
Mutation testing has been shown to bean effectivewayof testing software in terms
of fault revealing capacityand it has been widely explored. One advantage of
mutation testing is its flexibilityin the sense that it can beapplied to several scopes,
e.g, unit and integration testing in differentlanguages [17, 4], test of object oriented
XV Simpósio Brasileiro de Engenharia de Software
- 282 -
Table 2: Proposed set of mutantoperators
Operator name Meaning
DelSync Removes the synchronized attribute from amethod
declaration or removes asynchronized statement
DelStat Deletes astatementin asynchronized methodor block
MoveBrace Moves {and }up and down
ReplSyncObject Replaces synchronization object
DelWait Deletes acall to the wait method
ReplWait Replaces acall to atimed wait byacall to anon-timed
wait and vice-versa
IncrDecrWait Incrementand decrementargumentfor timed wait
DelNotify Deletes acall to notify or notifyAll
ReplNotify Replaces acall to notify byacall to notifyAll and
vice-versa.
ReplArg Replaces argumentwith constant
DelSyncCall Deletes acall to asynchronized method
SwitchArg Switcharguments in acall to synchronized method
ReplMeth Uses methodwith same name and other signature
InsNegArg Inserts unary (negation) operators in an argument
ReplTargObjReplaces the object in acall to synchronized method
programs [11] and test of formal specifications using differentlanguages [7, 8, 6, 20].
Toadapt the criterion to these differentscenarios it is necessary to identify the
characteristics one wants to exercise on that specific scopeand to develop mutant
operators for it.
This paper focuses on the definition of mutantoperators to exercise the features
of the Javalanguage related to concurrency and synchronization. Mutantoperators
for other concurrentenvironments havebeen defined [17, 19], but due to the differ-
ences between those environments and the Java environment, new mutantoperators
are required to address concurrency in Java. Apilot study [2] has been conducted
using well knowoperators, not specifically designed to this goal, and confirmed
this necessity.The mutantoperators herein presented tries to cover most of the
concurrency- and synchronization-related features of the Java environmentand still
becost effective, restricting the number of operators and mutants that they can
generate.
The application of mutation testing to concurrentprograms introduces addi-
tional problems. In particular it is necessary to deal with the intrinsic nondeter-
minism derived from concurrency.The execution of aprogram with agiven input
can produce differentcorrect behaviors. This constitutes aproblem in comparing
the results of executing mutants with the results of the original program. Atool
that supports mutation testing in this scenario must provide awayto assure that
twoexecutions of aprogram with the same test case lead to the same behavior.
Currently weare working on developing the tools necessary to experimentthe re-
sults described in this paper. The first tool applies the mutantoperators described
in this paper to automatically generate mutants. The second aims at recording
executions of the original program to execute the same synchronization sequences
XV Simpósio Brasileiro de Engenharia de Software
- 283 -
on its mutants.
6Acknowledgements
The authors would liketo thank CNPq, CAPES and FAPESP for partially sup-
porting this project.
References
[1] H. Agrawal, R. A. DeMillo, R. Hataway,Wm. Hsu, W. Hsu, E. Krauser, R. J.
Martin, A. P.Mathur, and E. H. Spafford. Design of MutantOperators for
CProgramming Language. TechReport SERC-TR41-P,Software Engineering
ResearchCenter, Purdue University,March1989.
[2] M. Delamaro, M. Pezz`e, A. M. R. Vincenzi, and J. C. Maldonado. Ap-
plying Mutation Testing to Multi-threaded JAVA Programs. TechReport,
www.din.uem.br/˜delamaro/papers/relat.ps.gz,2001.
[3] M. E. Delamaro. Proteum: Um Ambiente de Teste Baseado Na An´alise de
Mutantes.Master thesis, SCE-ICMSC-USP,S˜ao Carlos -SP,October 1993.
[4] M. E. Delamaro, J. C. Maldonado, and A. P.Mathur. Interface Mutation: An
Approachfor Integration Testing. IEEE Transactions on SoftwareEngineering,
27(3):228–247, March2001.
[5] R. A. DeMillo and A. J. Offutt. ConstraintBased Automatic Test Data Gener-
ation. IEEE Transactions on SoftwareEngineering,17(9):900–910, September
1991.
[6] S. C. PF. Fabbri. AAn´alise de Mutantes No Contexto de Sistemas Reativos:
Uma Contribui¸c˜ao ParaOEstabelecimento de Estrat´egias de Teste eValida¸c˜ao.
Doctoral dissertation, IFSC -USP,S˜ao Carlos -SP,October 1996.
[7] S. C. P.F. Fabbri, J. C. Maldonado, P.C. Masiero, and M. E. Delamaro.
Mutation Analysis Testing for Finite State Machines. In Proceedings of the 5th
International Symposium on SoftwareReliability Engineering (ISSRE),pages
220–229, Monterey -CA, November 1994.
[8] S. C. P.F. Fabbri, J. C. Maldonado, P.C. Masiero, and M. E. Delamaro.
Mutation Analisys Applied to Validate Specifications Based on Petri Nets. In
Proceeding of the 8th IFIP Conferenceon Formal Descriptions Techniques for
Distribute Systems and Communication Protocols,pages 329–337, Montreal,
October 1995.
[9] J. Gosling, B. Joy,G. Steele, and G. Bracha. Tha Java Language Specification.
JavaSeries. Addison-Wesley,2nd edition, June 2000.
[10] S. Kim, J. A. Clark, and J. McDermid. The Rigorous Generation of Java
Mutation Operators Using HAZOP.In Proceedings of the 12th International
Conferenceon Software&Systems Engineering and their Applications (IC-
SSEA’99),December 1999.
[11] S. Kim, J. A. Clark, and J. McDermid. Class Mutation: Mutation Testing for
Object-Oriented Programs. In Proceedings of the FMES,October 2000.
[12] K. N. King and A. J. Offutt. AFortran Language System for Mutation Based
Software Testing. Software-Practiceand Experience,21(7):685–718, July 1991.
XV Simpósio Brasileiro de Engenharia de Software
- 284 -
[13] A. P.Mathur and W. E. Wong. Evaluation of the Cost of Alternate Muta-
tion Strategies. In Proceedings of the 7th Brazilian Symposium on Software
Engineering,pages 320–335, Rio de Janeiro, RJ, Brazil, October 1993.
[14] A. J. Offutt. Coupling Effect: Fact or Fiction. In Proceedings of the 3rd
Symposium on SoftwareTesting, Analysis, and Verification (ISSTA’89),pages
131–140, Key West, FL, December 1989.
[15] A. J. Offutt and J. H. Hayes. ASemantic Model of Program Faults. In
Proceedings of the International Symposium on SoftwareTesting and Analisys
and Verification (ISSTA’96),San Diego, CA, 1996.
[16] A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An Experimental
Determination of SufficientMutantOperators. ACM Transactions on Software
Engineering Methodology,5(2):99–118, 1996.
[17] A. J. Offutt, J.M. Voas, and J. Payne. Mutation Operators for ADA. Tech-
nical Report ISSE-TR-96-09, Departmentof ISSE, George Mason University,
Fairfax, VA, March1996.
[18] R. L. Probert and F. Guo. Mutation Testing of Protocols: Principles and Pre-
liminary Experimental Results. In IFIP TC6 –ThirdInternational Workshop
on Protocol Test Systems,pages 57–76. North-Holland, 1991.
[19] S. Silva-Barradas. Mutation Analysis of Concurrent Software.Phd thesis,
Departmentof Eletronic and Informatics, Polythecnic of Milan, Milan, Italy,
1997.
[20] S. R. S. Souza, J. C. Maldonado, S. C. P.F. Fabbri, and W. Lopes de Souza.
Mutation Testing Applied to Estelle Specifications. In 33rdHawaii Interna-
cional Conferenceon System Sciences, Mini-Tracks: DistributedSystems Test-
ing,Maui, Hawai, January 2000. (Accepted for publication in aspecial issue
on Distributed Systems Testing of the Software QualityJournal).
[21] A. M. R. Vincenzi, J. C. Maldonado, E. F. Barbosa, and M. E. Delamaro.
Unit and Integration Testing for CPrograms Using Mutation-based Criteria.
Journal of SoftwareTesting, Validation and Reliability,2001. to appear.
XV Simpósio Brasileiro de Engenharia de Software
- 285 -
