 |
BACTERIO Mutation Test System is a testing tool for Java based on the mutation technique. Mutation is a testing technique which is based on the capability of test cases to find faults on the system under test. Mutation has been developed in the research field during the last decades, and its transference to the industrial field is being done successfully in recent years.
|
|
|
What is mutation testing? |
|
Being S the system under test, mutation generates a set of mutants M={M1, M2, … Mn} from S, each one containing one or more syntactic changes, most of which will be faults. The original idea of mutation is to build a test suite that finds all the faults injected into the mutants. Each syntactic change is seeded into the mutant using a "mutation operator":
- Table a) shows the source code of the original program and some mutants.
- Table b) presents the results from the execution of some test cases against different program versions.
The test case corresponding to test data (1, 1) produces different outputs for the original program (whose output is correct) and for the Mutant 1: thus, this test case found the fault introduced in the mutant, and it is said that the mutant is killed. On the other hand, since all the test cases give the same output as with the original program, it is said that Mutant 4 is alive. Moreover, this mutant will never be killed by any test case since it is "functionally equivalent" to the original code. Functionally-equivalent mutants may be considered as noises in the step of result analysis, since they impede knowing the quality of the test suite: they have a fault (actually, a syntactic change on the original code) that is impossible to find by any test case.
|
Version
|
Code
|
|
Original
|
int sum(int a, int b) {
return a + b;
}
|
|
Mutant 1
|
int sum(int a, int b) {
return a - b;
}
|
|
Mutant 2
|
int sum(int a, int b) {
return a * b;
}
|
|
Mutant 3
|
int sum(int a, int b) {
return a / b;
}
|
|
Mutant 4
|
int sum(int a, int b) {
return a + b++;
}
|
a) Code of some mutants
|
|
Test data (a,b)
|
|
(1, 1)
|
(0, 0)
|
(-1, 0)
|
(-1, -1)
|
|
Program versions
|
Original
|
2
|
0
|
-1
|
-2
|
|
Mutant 1
|
0
|
0
|
-1
|
0
|
|
Mutant 2
|
1
|
0
|
0
|
1
|
|
Mutant 3
|
1
|
Error
|
Error
|
1
|
|
Mutant 4
|
2
|
0
|
-1
|
-2
|
b) Results of some test data
|
|
|
What is BACTERIO Mutation Test System? |
|
BACTERIO Mutation Test System is a tool for testing complete Java systems. It generates copies of the application and introduces one or more faults in each one. Then, a test suite consisting of JUnit or UISpec test cases can be executed against the original and the mutants versions. When a test suite doen't find faults in the original system but it does find the seeded faults in the mutant versions, it can be assured that the system has been successfully tested, since the test suite is complete, it doesn't detect faults on the original system, and it kills a high percentage of mutants.
The next figure sums up the mutation-based testing process with the tool BACTERIO Mutation test System:
The testing process with BACTERIO Mutation Test System comprises the following steps:
1. Mutant generation
In this first step, BACTERIO Mutation Test System generates faulty copies (mutants) of the classes of the system under test. In order to do that, it provides several mutation operators: arithmetic operator replacement, parameter swapping, operator increase, operator decrease, etc.
2. Mutant combination
In this second step, BACTERIO Mutation Test provides three algorithms (one mutant per version, using each mutant at least once, and using every pair of mutants) to combine the mutants and generate faulty versions.
3. Test Case Execution
In the last step, BACTERIO Mutation Test System executes the test cases against the different faulty versions. Then, it provides the percentage of killed mutants, and, therefore, the coverage reached with the designed test suite.
|
|
BACTERIO Mutation Test System demo version |
|
|
Products 
