Metamorphic testing

src: i.ytimg.com

Metamorphic testing (MT) is a software testing technique that attempts to alleviate the test oracle problem. A test oracle is the mechanism by which a tester can determine whether a program has failed. A test oracle problem occurs when it is difficult to determine the expected outcomes of selected test cases or to determine whether the actual outputs agree with the expected outcomes.

MT recommends that, given one or more test cases (called "source test cases" or "original test cases") and their expected outcomes (if any), one or more follow-up test cases can be constructed to verify the necessary properties (called "metamorphic relations" or "MRs") of the system or function to be implemented. Consider, for example, a program that implements sin x correct to 100 significant figures. A useful metamorphic relation for sine functions is "sin (? - x) = sin x", Thus, even though the expected value of sin x₁ for the source test case x₁ = 1.234 correct to the required accuracy is not known, a follow-up test case x₂ = ? - 1.234 can be constructed. We can verify whether the actual outputs produced by the program under test from the source test case and the follow-up test case are consistent with the MR in question. Any inconsistency indicates a failure of the implementation.

In general, an MR does not need to be an equation but can be a relation. Metamorphic testing can be applied to any problem in which a necessary property involving multiple executions of the target function can be formulated. Some examples of successful applications include partial differential equations, ubiquitous computing, services computing, imaging software, bioinformatics, software product lines, and web search engines.

Metamorphic testing has also been applied to machine learning, AI algorithms, and deep learning models based on neural networks . It is believed that this is a promising approach to validate machine learning software .