Mutation testing is a mature technology for testing data quality assessment in traditional software.
Mutation testing is a form of white-box testing.
Mutation testing (or mutation analysis or program mutation) is used to design new software tests and evaluate the quality of existing software tests.
The goals of mutation testing are multiple:
- identify weakly tested pieces of code (those for which mutants are not killed)
- identify weak tests (those that never kill mutants)
- compute the mutation score
- learn about error propagation and state infection in the program
Selecting some mutation operations, and applying them to the source code for each executable code segment in turn.
The result of using a mutation operation on a program is called a mutant heterogeneity.
If the test unit can detect the error (ie, a test fails), then the mutant is said to have been killed.
def foo(x: int, y: int) -> int:
Given some test cases, we find that unit test cannot find variants
assertEquals(2, foo(2, 2))
assertEquals(0, foo(2, 1))
assertEquals(0, foo(-1, 2))
Add new tests to achieve the effect of eliminating variants:
assertEquals(0, foo(2, 0))
def bar(a: int, b:int) -> int:
Given a test case that will absolutely pass:
assertEquals(1, foo(1, 1))
But we need to kill the mutation by adding more test cases:
assertEquals(0, foo(1, 0))
assertEquals(0, foo(0, 1))
assertEquals(1, foo(0, 0))
In deep learning, you can also create variants by changing the operators in the model.
Adding the idea of the mutation test to the deep learning model, if the performance of the model after the mutation is unchanged, then there is a problem with the test set
It is necessary to add or generate higher quality test data to achieve the data enhancement effect.