CN113778865A - Self-adaptive random generation method for test case of metamorphic test - Google Patents

Abstract

A self-adaptive generation method for a test case of metamorphic testing comprises the following steps: (1) initializing; (2) checking whether all partitions of all parameters are covered; (3) selecting uncovered partitions, and randomly generating a metamorphic test case set; (4) calculating the distance between the initial input and the subsequent input; (5) taking a group of metamorphic test inputs with the largest distance as candidate inputs; (6) calculating an anomaly score for the candidate input relative to the result set; (7) and (5) removing the candidate input with the abnormal score smaller than the threshold value, if the candidate input set is empty, returning to the step (3), and if not, returning to the step (2). The invention supports metamorphic testing, and automatically generates a test case until all input partitions at least contain one test input; the method supports the difference measurement of initial test input and subsequent test input, supports the difference measurement of test input and result set, and improves the possibility of finding defects and supports small data sets, and the measurement result is not influenced by parameter dimension.

Description

Self-adaptive random generation method for test case of metamorphic test

Technical Field

The invention relates to a self-adaptive random generation method of a test case, in particular to a self-adaptive random generation method of a test case of metamorphic testing.

Background

Scientific calculation and industrial design software such as numerical simulation nuclear design and safety analysis software, aeronautical engine dynamic design simulation software and the like are adopted, and the problem that an expected result is difficult to construct or the construction cost is extremely high generally exists because complicated partial differential equations need to be solved and no analytic solution exists, so that the problem is called the Oracle problem of software testing. The traditional test method verifies the tested program by directly comparing an actual result with an expected result, so that the Oracle problem makes it difficult to implement sufficient test on the software, the sufficient test is lacked, and the software quality is difficult to ensure.

Metamorphic Testing (MT) is one of the currently accepted effective methods for solving the Oracle problem, and it indirectly performs verification by examining whether Metamorphic Relations (MR) are satisfied between input and output when a program is executed multiple times. For example, assuming that the program under test p implements the sine function sin, the relationship given by the periodic, available degeneracy of sin: sin (x) ═ sin (x +2 pi), a random value x1 is generated as an initial Test Case (Source Test Case, STC), x2 is derived from x1+2 pi as a subsequent Test Case (Follow-up Test Case, FTC), p (x1) is compared with p (x2), if equal, p passes the Test, otherwise p fails the Test.

The Metamorphic Test Case (MTC) is composed of a group of Test cases (STC, FTC), when the Metamorphic relation is a binary relation, the MTC comprises an STC and an FTC, when the Metamorphic relation is an n-element relation, the MTC comprises an STC and n-1 FTCs, and obviously, the execution times of Metamorphic tests are higher than those of the traditional Test method. In order to reduce the testing cost and improve the testing effectiveness, it is very important to select a test case with higher possibility of identifying defects.

Research shows that the software defect distribution has aggregation, and three types of software defect distribution modes under a two-dimensional input domain are shown in the attached figure 1.

If a test case does not find a defect, then a test case that is far away from it will be easier to detect a defect than a nearby test case. Therefore, the most important problem of the test case generation method is to keep the test case difference as much as possible.

The test case generation method mainly comprises a special value method and a random value method, wherein the special value method needs the tester to have richer field knowledge of the tested software, and the random value method does not depend on prior knowledge and is widely applied to metamorphic testing, but the test coverage rate cannot be effectively improved.

An Adaptive Random Testing (ART) is inspired by a defect distribution mode, and a candidate test case which is the largest distance from an executed test case is selected from randomly generated test input. However, this method has three disadvantages: (1) the existing self-adaptive random generation method does not support the difference measurement of the structure; (2) for a program with multiple input parameters (high-dimensional input), the distance calculation amount is large; (3) when there is a large difference in the dimension of the input parameter, the calculation result based on the distance or density depends on the parameter with a large value, such as n-dimensional input { x1, x2, …, xn }, and the value ranges of x1 and x2 are (0,1) (-100,10000), respectively, then the influence of x2 on the distance is significantly greater than that of x1, and is easily influenced by the dimension of the input parameter.

Disclosure of Invention

The invention aims to solve the technical problem of overcoming the defects of the prior art and provide a test case self-adaptive random generation method which has a wide application range and supports metamorphic testing without being influenced by dimensions.

The technical scheme adopted by the invention for solving the technical problem is that the self-adaptive random generation method of the test case of the metamorphic test comprises the following steps:

the method comprises the following steps: initialization: the method comprises the steps of firstly analyzing an input field according to a design document of the tested program, identifying an equivalence class, dividing the input field into n disjoint areas according to the equivalence class, wherein the input field is divided into n disjoint areas according to the equivalence class, for example, the input X consisting of n parameters is { X1, X2.., xn }, and the input field of X1 is [0,100 []Are divided into [0,20 ], [20,50 ], [50,100]Three areas, other input parameters and so on, initializing result set C_RTC(Result Test Case, RTC), if there is an executed Test Case, copy to C_RTCOtherwise, randomly generating a batch of input C according to the input field_RTC；

Step two: check if all partitions for all parameters have been covered: stopping the generation process if all the partitions of all the input parameters of the program under test are covered by at least one test input, and outputting C_RTCOtherwise, executing the next step;

step three: selecting uncovered partitions, and randomly generating a metamorphic test case set: randomly generating k MTC, and establishing an MTC set C_MTC；

Step four: calculating the distance of the initial input from the subsequent input: calculating C_MTCThe distance D between the STC of each MTC and the FTC_sf；

Step five: taking a group of MTC with the largest distance as a candidate test input set C_CTC(Candidate Test Case，CTC)；

Step six: computing each MTC relative result set C using an isolated forest algorithm_RTCIs given an abnormality score S_A(Anomaly Score)；

Step seven: from C_CTCRemoving MTC with score less than threshold when if C_CTCIf the test result is null, returning to the step three, otherwise inputting the candidate testC_CTCPut into result set C_RTCAnd returning to the step two.

Further, in the third step, the method for establishing the metamorphic test case set comprises the following steps: randomly selecting one partition from each input parameter to generate an initial Test input (Source Test Case, STC), obtaining a subsequent Test input (Follow-up Test Case, FTC) according to a Metamorphic relation, forming a Metamorphic Test Case (MTC) by the two, and so on.

Further, in the fourth step, the distance D is calculated_sfThe formula of (1) is:

where n is the number of input parameters and i is the input parameter number.

Further, in the sixth step, an abnormality score S is calculated_AThe formula of (1) is:

wherein x is MTC, using C_RTCConstructing an orphan forest as a training set, h (x) is the height of x in the orphan tree, E (h (x)) is the average of h (x), C (n) is C_RTCThe average height of the isolated tree when the test input equals n, for normalizing h (x), S_AA closer to 1 indicates a larger degree of abnormality.

The invention has the following positive effects:

(1) supporting metamorphic testing, and automatically generating test cases until all the input domain partitions are covered by at least one test case;

(2) supporting the difference measurement of metamorphic test cases, using the distance D between STC and FTC_sfThe difference degree of a single MTC is expressed, the fact that most of the MTC is binary MR and only comprises one FTC is considered, so that the calculation cost is low, and only two test cases participate in calculation, so that the dimension influence is low;

(3) supporting the difference measurement of the test cases relative to the result set, and measuring the candidate metamorphic test cases relative to the result set C by adopting the abnormal score of the isolated forest algorithm_RTCThe method calculates the complexThe complexity is O (n), the method has linear time complexity, high speed in processing high-dimensional data and low memory consumption, and is commonly used for network attack detection, financial transaction fraud detection and the like;

(4) calculating candidate test input and result set C without influence of input parameter dimension_RTCWhen the difference is different, a distance or density-based method is not used, so the method is not influenced by the dimension;

(5) the probability of finding the defects is improved, through two rounds of difference measurement, test inputs with obvious differences are automatically selected on the basis of a random value method, and according to a defect distribution model, the test inputs have higher probability of finding the defects;

(6) the method supports small data sets, and due to the cost, the number of test cases, especially metamorphic tests, is expected to be reduced as much as possible on the premise of keeping the error detection capability, so that a result set is most likely to be the small data sets, and researches show that the detection effect of the isolated forest algorithm on the small data sets is better.

Drawings

FIG. 1 is a schematic diagram of three types of software defect patterns in software engineering practice;

FIG. 2 is a flow chart of an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples.

With reference to fig. 2, taking the sine function sin as an example, the implementation comprises the following steps,

(1) initialization: suppose that the field 0,2 pi is to be input]Is divided into [0,1 ]]、(1,π]、(π,5]、(5,2π]Assuming no executed test cases exist, randomly generating a batch of input C according to the input field_RTC＝Random(0,2π,10)＝{π/3,2,3,...,5π/3}；

(2) Check if all partitions for all parameters have been covered: if all the partitions of all the parameters at least comprise one test case, stopping the test if the detection result is true, and otherwise, executing the next step;

(3) selecting uncovered partitions, and randomly generating a metamorphic test case set: in [0,1 ]]Interval random generation of 10 initial test inputs C_STCTransmutation relation MR periodically obtained from sin (0,1,10) {0.1,0.2₁：x₂＝x₁+2π，sin(x₂)＝sin(x₁) Generating a subsequent test input C_FTC＝{C_stc+2 pi } - {0.1+2 pi, 0.2+2 pi, · 1+2 pi }, both of which constitute C_MTC{ (0.1,0.1+2 pi), (0.2,0.2+2 pi),. ·, (1,1+2 pi) }, and so on for other partitions;

(4) calculating the distance of the initial input from the subsequent input: calculating C_MTCDistance of STC to FTC, D, of each MTC in_sf(MTC₁)＝√(0.1-(0.1+2π))²2 pi, and so on for other MTC;

(5) taking a group of MTC with the largest distance as a candidate test input set C_CTCAssuming that the number of elements is taken to be 3, then from C_MTCGet D_sfMaximum 3 MTC Components C_CTC＝{MTC₁,MTC₂,MTC₃}；

(6) Computing each MTC relative result set C using an isolated forest algorithm_RTCIs given an abnormality score S_AIs provided with S_A(MTC₁)＝0.98,S_A(MTC₂)＝0.50,S_A(MTC₁)＝0.43；

(7) From C_CTCRemoving MTC with score less than threshold, assuming abnormal score threshold of 0.85, removing C after MTC_CTC＝{MTC₁}，C_CTCNot null, input candidate test C_CTCPut into result set C_RTCAnd returning to the step 2, otherwise, returning to the step 3.

The embodiment can embody the difference between the initial input and a plurality of subsequent inputs in the metamorphic test case; evaluating the difference between the candidate test cases and the executed test case set by using an isolated forest anomaly detection algorithm; the calculation complexity is reduced to linear complexity, and the method has engineering practicability.

Various modifications and variations of the present invention may be made by those skilled in the art, and they are still within the scope of the present patent invention provided they are within the scope of the claims and their equivalents.

What is not described in detail in the specification is prior art that is well known to those skilled in the art.

Claims (4)

1. A self-adaptive random generation method for a test case of metamorphic testing is characterized by comprising the following steps:

the method comprises the following steps: initialization: firstly, analyzing an input field according to a design document of the tested program, identifying an equivalence class, dividing the input field into n disjoint areas according to the equivalence class, wherein an input X formed by n parameters is { X1, X2, …, xn }, and an input field of X1 is [0,100 }, and each parameter has different value ranges]Are divided into [0,20 ], [20,50 ], [50,100]Three areas, other input parameters and so on, initializing result set C_RTC(Result Test Case, RTC), if there is an executed Test Case, copy to C_RTCOtherwise, randomly generating a batch of input C according to the input field_RTC；

Step two: check if all partitions for all parameters have been covered: stopping the generation process if all the partitions of all the input parameters of the program under test are covered by at least one test input, and outputting C_RTCOtherwise, executing the next step;

step three: selecting uncovered partitions, and randomly generating a metamorphic test case set: randomly generating k MTC, and establishing an MTC set C_MTC；

Step four: calculating the distance of the initial input from the subsequent input: calculating C_MTCThe distance D between the STC of each MTC and the FTC_sf；

Step five: taking a group of MTC with the largest distance as a candidate test input set C_CTC(Candidate Test Case，CTC)；

Step six: computing each MTC relative result set C using an isolated forest algorithm_RTCIs given an abnormality score S_A(Anomaly Score)；

Step seven: from C_CTCRemoving MTC with score less than threshold when if C_CTCIf the test result is empty, returning to the step three, otherwise inputting the candidate test into the test C_CTCPut into result set C_RTCReturn stepAnd step two.

2. The method of claim 1, wherein the method comprises: in the third step, the method for establishing the candidate metamorphic test case set comprises the following steps: randomly selecting one partition from each input parameter to generate an initial Test input (Source Test Case, STC), obtaining a subsequent Test input (Follow-up Test Case, FTC) according to a Metamorphic relation, forming a Metamorphic Test Case (MTC) by the two, and so on.

3. The method for adaptively and randomly generating test cases for metamorphic testing according to claim 1 or 2, wherein: in the fourth step, the distance D is calculated_sfThe formula of (1) is:

where n is the number of input parameters and i is the input parameter number.

4. The method for adaptively and randomly generating test cases for metamorphic testing according to claim 1 or 2, wherein: in the sixth step, an abnormality score S is calculated_AThe formula of (1) is:

wherein x is MTC, using C_RTCConstructing an orphan forest as a training set, h (x) is the height of x in the orphan tree, E (h (x)) is the average of h (x), C (n) is C_RTCThe average height of the isolated tree when the test input equals n, for normalizing h (x), S_AA closer to 1 indicates a larger degree of abnormality.

Images