Keywords: Mutation operator; Production rule; Software testing; Mutation Testing;
Before understanding mutation operators in detail, it is important to understand the concept of software fault type. A software fault type is a textual description of a specific kind of fault that can occur in any program. A fault type can have many instances in the program depending on where and what change is made. For example, “changing arithmetic operator” is a software fault type that can exist in both constructor and a method of a program, creating two distinct faults in a program. Each software fault type can be represented by one or more mutation operators. Each mutation operators induces a specific software fault in the program. Furthermore, a single mutation operator can induce multiple instances of a fault in a program and in consequence inducing multiple instances of the software fault type. A mutation operator definition describes how to insert an instance of a specific fault type in a program. Multiple faulty versions of the programs, namely “mutant” programs, are generated based on where and what changes are made in a program using the mutation operator. A mutation operator may not necessarily generate all instances of a software fault type. As the mutation operator should induce a specific fault in the program, there is a need for a formal definition that precisely defines the changes that will be made in the program by the mutation operator representing a specific software fault.
Literature research shows the lack of formalization of mutation operator definitions. It shows that mutation operators usually have been defined using a sample code or text only. In this paper, the author presents a grammar to formalize the mutation operator definitions that can be applied to a program to precisely induce a particular fault and analyze the behavior of the program
Kim et al., Ma et al., and Derezińska defined mutation operators for OO programs [6,11,13,14]. Specifically, Kim et al. introduced a Class Mutation method that targeted fault types that could occur in OO programs [11]. This research categorized fault types based on polymorphic types, method overloading, method overriding, field variable hiding, information hiding, static/dynamic stated of objects and exception handling. Further, Ma et al. defined more mutation operators for OO programs and evaluated mutation testing for OO programs using empirical studies [13,14]. Research by Ma et al. defines mutation operators specifically for Java language, but a majority of operators can be applied to other OO programming languages [13,14]. Offutt et al. discussed the faults and mutation operators based on OO properties, such as polymorphism and Inheritance [16,17].
a) A finite set of production rules – These production rules are used to create a formal specification for the mutation operators that can be applied to a program to generate mutants for a specific fault. The symbol PR represents the production rule for the mutation operator. Figure 1 shows the grammar production rules with start symbol S.
b) A finite set of nonterminal symbols – The start symbol in this grammar is PR with nonterminal symbols A, B, C, E, F, G, H, K, L, and S1.
c) A finite set of terminal symbols – The keywords and the operators are the terminal symbols in this grammar. The language-specific keywords are also included in this set of terminal symbols. D, I, J, OP, OP1 and Z are terminal symbols.
For this experiment, two programs were used – Geometric Objects and Banking. Geometric Objects program calculated the perimeter and area of various geometric objects and banking created different types of accounts and defines operations performed on these accounts. Even though these programs are simple but they addressed the properties of the object-oriented programming languages. These programs were written using Java programming language and therefore Java mutant programs were created but the MOPRS is generic and can be applied to any programming language. As a part of the research, a program was written to read these specifications of a mutation operator from
Text definition: If there is an overloaded method with some parameters, then change the contents of the original method to the nth overloaded method call.
Production Rule: If there are N number of overloaded methods with some parameters, x(y, z), x(y), x (z), Then change the contents of x(y, z) to a method call for x(y) and x (z) one at a time.
Text definition: If there is another method call inside the body of the method, then change the call to another compatible method.
Production Rule: If there is a method call x() inside the body of method z. Then change the method call x() to y() in the body of the method z, where x and y are compatible.
This research created mutation operators for fault types such as assignment that violates a state invariant (IVS) and Method Incorrectly Performed (MIP). 11 mutation operators were specified in the first category and 4 for the second type. Also, a test suite was created for testing the programs induced with these mutation operators [8]. All the mutants for both fault types were killed by the test suite.
All mutation operators were induced and tested by the test suite and the suite was successful in finding the faults in the program Thus, we can say that the new and the improved mutation operators were effective in creating mutants that were killed i.e. found by the test suite and are a good representation of faults that can be present in a program for that fault type. There is still a need to address other fault types listed in table 1. There is need for new mutation operators that represent specific faults. The future work includes creating new mutation operators for the existing fault types and also creating new fault types that are missing from the existing collection.
Fault types associated with Classes |
Related Mutation Operators |
Incorrect initialization or missing re-initialization |
JDC (Java-supported default constructor create) |
Fault types associated with Attributes |
Related Mutation Operators |
Incorrect values of attributes |
ISK (super keyword deletion) |
Not initialized |
JID (Member variable initialization) deletion) |
Incorrect visibility, access modifier misuse |
AMC (Access modifier change) |
Fault types associated with Methods |
Related Mutation Operators |
Incorrect method used |
EAM (Accessor method change), EMM (Modifier method change) |
Parameter mismatch |
OAO (Argument order change) |
Fault types Associated with Inheritance |
Related Mutation Operators |
Deletion of resource in subclass violates abstraction of superclass |
IHD (Hiding variable deletion), IHI (Hiding variable insertion) |
Incompatibility between subclass resources and existing superclass resources. |
IOP (Overridden method calling position change) |
Incorrect initialization (super not used for initialization) |
IPC (Explicit call of a parent’s constructor deletion) |
Inheritance allows super class features with the same names, providing the opportunity for misnaming. Changes to either superclass will affect the subclass. |
IOR (Overridden method rename) |
Original resource in super class not overridden in subclass |
IOD (Overriding method deletion) |
super keyword misuse |
ISK (super keyword deletion) |
Fault types associated with Polymorphism |
Related Mutation Operators |
Failure to check the compatibility of the actual parameters |
PPD (Parameter variable declaration with child class type) |
Failures associated with instantiation |
PNC (new method call with child class type), PMD (new method call with child class type) |
Fault types associated with Method |
Related Mutation Operators |
Misusing a dynamically bound method |
OMD (Overloading method deletion) |
Message not received by correct method or message received by incorrect method |
OMD (Overloading method deletion), OAO (Argument order change), OAN (Argument number change) |
Method incorrectly performed |
OMR (Overloading method contents change) |
Fault types associated with Inheritance manifested by polymorphism |
Related Mutation Operators |
State visibility anomaly |
SVA (State visibility anomaly) |
State definition inconsistency due to state variable hiding |
SDIH (State definition inconsistency hiding) |
State definition anomaly |
SDA (possible post-condition violation) |
State defined incorrectly |
SDI (State defined incorrectly) |
Inconsistent type use |
ITU (Inconsistent type use) |
Incomplete construction |
IC (Incomplete construction) |
Indirect inconsistent state definition |
IISD (Indirect inconsistent state definition) |
Anomalous construction behavior |
ACB (Anomalous construction behavior) |
Fault types associated with programming language specific features |
Related Mutation Operators |
this keyword misuse |
JTD |
Static modifier misuse |
JSC |
- Agrawal H, DeMillo RA, Hathaway B, William H, Wynne H, Krauser EW, et al, Design of Mutant Operators for the C Programming Language. SERCTR41-P. 1989.
- Alexander RT, Offutt J, Bieman JM. Syntactic fault patterns in Object-oriented programs. Proceedings of the Eighth IEEE International Conference on Engineering of Complex Computer Systems. 2002:193-202.
- Binder RV. Testing Object-oriented Software: A Survey. Journal of Software Testing, Verification and Reliability. 1996;6:125-252.
- Binder RV. Testing Object-oriented Systems: models, patterns, and tools. Addison Wesley. 1999.
- DeMillo RA, Lipton RJ, Sayward FG. Hints on test data selection: help for the practicing programmer. Computer. 1978;11(4):34-41.
- Derezińska A. Advanced mutation operators applicable in C# programs. Software Engineering Techniques: Design for Quality. 2006:283-288.
- Firesmith DG. Testing Object-oriented Software. Proceedings of the Eleventh International Conference on Technology of Object-oriented Languages and Systems. 1993:407-426.
- Gupta P, Gustafson DA. Object-oriented Testing beyond Statement Coverage. Proceedings of International Conference on Computer Applications in Industry and Engineering, 2010:150-155.
- Hayes HJ. Testing of Object-oriented Programming Systems (OOPS): A Fault-Based Approach. Proceedings of Object-oriented Methodologies and Systems. 1994:205-220.
- Jia Y, Harman M. MILU: A Customizable, Runtime-Optimized Higher Order Mutation Testing Tool for the Full C Language. Proceedings of Third Testing: Academic and Industrial Conference Practice and Research Techniques. 2008:94-98.
- Kim S, Clark JA, McDermid JA. Class mutation: Mutation testing for Object-oriented programs. Proceedings of Net Object days Conference on Object-oriented Software Systems. 2000.
- Lin JC, Huang YL, Liu CH. Testability Analysis for Polymorphism. Proceedings of the Second IASTED International Multi-Conference on Automation, Control, and Information Technology. 2005:98-103.
- Ma YS, Kwon YR, Offutt J. Inter-class mutation operators for Java. Proceedings of the 13th International Symposium on Software Reliability Engineering. 2002:352-363.
- Ma YS, Harrold MJ, Kwon YR. Evaluation of Mutation Testing for Object-oriented programs. Proceedings of the 28th international conference on Software engineering, 2006:869-872.
- Offutt JA, Lee A, Rothermal G, Untch RH, Zapf C. An experimental determination of sufficient mutant operators. ACM Transactions of Software Engineering Methodology. 1996;5(2):99-118.
- Offutt JA, Untch RH. Mutation 2000: Uniting the orthogonal. In Mutation testing for the new century. 2001:34-44.
- Offutt JA, Alexander R, Wu Y, Xiao Q, Hutchinson C. A fault model for subtype Inheritance and Polymorphism. Proceedings of the 12th International Symposium on Software Reliability Engineering. 2001:84-93.
- Purchase AJ, Winder RL. Debugging Tools for Object-oriented Programming. Journal of Object-oriented Programming, 1993;4(3):10-27.
- Walkinshaw N, Roper M, Wood M. Collecting and Categorizing Faults in Object-oriented Code. Sheffield: UKTest. 2005.
- Woodward MR. Mutation Testing - an Evolving Technique. Proceedings of IEE Colloquium on Software Testing for Critical Systems. 1990.
- Chevalley P. Applying mutation analysis for Object-oriented programs using a reflective approach. Proceedings of the 8th Asia-Pacific Software Engineering Conference (APSEC). 2001.