CN113778865B - Test case self-adaptive random generation method for metamorphic test - Google Patents
Test case self-adaptive random generation method for metamorphic test Download PDFInfo
- Publication number
- CN113778865B CN113778865B CN202110994979.9A CN202110994979A CN113778865B CN 113778865 B CN113778865 B CN 113778865B CN 202110994979 A CN202110994979 A CN 202110994979A CN 113778865 B CN113778865 B CN 113778865B
- Authority
- CN
- China
- Prior art keywords
- input
- test
- metamorphic
- mtc
- rtc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Abstract
A test case self-adaptive generation method for metamorphic test comprises the following steps: (1) initializing; (2) Checking whether all partitions of all parameters are covered; (3) Selecting uncovered partitions, and randomly generating an metamorphic test case set; (4) calculating the distance between the initial input and the subsequent input; (5) Taking a group of metamorphic test inputs with the largest distance as candidate inputs; (6) Calculating an anomaly score for the candidate input relative to the result set; (7) Removing the candidate inputs with the abnormal score smaller than the threshold value, returning to the step (3) if the candidate input set is empty, and returning to the step (2) otherwise. The invention supports metamorphic test, and automatically generates test cases until all input partitions at least contain one test input; the method supports the difference measurement of the initial test input and the subsequent test input, supports the difference measurement of the test input and the result set, and the measurement result is not influenced by the parameter dimension, so that the possibility of finding defects is improved, and a small data set is supported.
Description
Technical Field
The invention relates to a test case self-adaptive random generation method, in particular to a test case self-adaptive random generation method for metamorphic test.
Background
The method adopts numerical simulation nuclear design, safety analysis software, aero-engine power design simulation software and other scientific calculation and industrial design software, and generally has no analytical solution because of the need of solving complex partial differential equations, and commonly has the problems that the expected result is difficult to construct or the construction cost is extremely high, which is called Oracle problem of software test. The conventional test method adopts a mode of directly comparing an actual result with an expected result to verify a tested program, and the Oracle problem makes it difficult to implement sufficient test on the software, lacks sufficient test, and makes the quality of the software difficult to guarantee.
The metamorphic test (metamorphic testing, MT) is one of the currently accepted effective methods of solving the Oracle problem, and indirectly performs verification by examining whether the metamorphic relationship (metamorphic relation, MR) is satisfied between the input and output when the program is executed multiple times. For example, assuming that the tested program p implements a sine function sin, the slough relationship is available from the periodicity of sin: sin (x) =sin (x+2pi), generating a random value x1 as an initial Test Case (STC), obtaining x2 from x1+2pi as a subsequent Test Case (FTC), comparing p (x 1) with p (x 2), if equal, passing the Test, otherwise, failing the Test.
The metamorphic test cases (Metamorphic Test Case, MTC) are composed of a group of test cases (STC, FTC), when the metamorphic relation is binary relation, the MTC comprises one STC and one FTC, when the metamorphic relation is n-ary relation, the MTC comprises one STC and n-1 FTC, and obviously, the metamorphic test execution times are higher than that of the traditional test method. In order to reduce the test cost and improve the test effectiveness, it is important to select test cases with higher probability of identifying defects.
The study shows that the software defect distribution has aggregation, and three types of software defect distribution modes under a two-dimensional input domain are shown in figure 1.
If one test case does not find a defect, then a test case that is far from it is easier to detect a defect than a nearby test case. Therefore, the most important problem of the test case generating method is to maintain the test case variability as much as possible.
The test case generation method mainly comprises a special value method and a random value method, wherein the former needs to have relatively rich field knowledge of tested software by a tester, and the latter is wider in application in metamorphic test because of not relying on priori knowledge, but cannot effectively improve the test coverage rate.
The adaptive random test (Adaptive Random Testing, ART) is inspired by a defect distribution pattern, and the test case with the largest distance from the executed test case is selected from randomly generated test inputs as a candidate test case. However, this method has three disadvantages: (1) The MTC which does not support one metamorphic test case comprises a plurality of test inputs, and the conventional self-adaptive random generation method does not support the difference measurement of the structure; (2) For a program having a plurality of input parameters (high-dimensional input), the distance calculation amount is large; (3) When there is a large difference in the dimensions of the input parameters, the calculation result based on the distance or the density depends on the parameters with large values, such as n-dimensional input { x1, x2, …, xn }, the value ranges of x1 and x2 are (0, 1), (-100,10000), respectively, and the influence of x2 on the distance is significantly greater than x1, and is easily influenced by the dimensions of the input parameters.
Disclosure of Invention
The invention aims to solve the technical problem of overcoming the defects of the prior art and providing a test case self-adaptive random generation method supporting metamorphic test, which has wide application range and is not influenced by dimension.
The invention solves the technical problems by adopting a technical scheme that the self-adaptive random generation method of the test case of the metamorphic test comprises the following steps:
step one: initializing: the tested program includes several input parameters, each parameter has different value ranges, firstly according to the design document of the tested program, analyzing input domain, identifying equivalence class, according to equivalence class dividing input domain into n non-intersecting regions, for example, the input X= { X1, X2, & gt, xn }, the input domain of X1 is [0,100 ]]Divided into [0, 20), [20, 50), [50,100 ]]Three regions, other input parameters, and so on, initialize result set C RTC (Result Test Case，RTC)Copy to C if there are executed test cases RTC Otherwise, randomly generating a batch of input C according to the input domain RTC ；
Step two: check if all partitions of all parameters have been covered: stopping the generation process if all partitions of all input parameters of the tested program are covered by at least one test input, and outputting C RTC Otherwise, executing the next step;
step three: selecting uncovered partitions, and randomly generating an metamorphic test case set: randomly generating k MTCs and establishing an MTC set C MTC ；
Step four: calculating the distance between the initial input and the subsequent input: calculation C MTC Distance D between STC and FTC of each MTC sf ；
Step five: taking the MTC group with the largest distance as a candidate test input set C CTC (Candidate Test Case，CTC)；
Step six: computing each MTC relative to the result set C using an isolated forest algorithm RTC Is of anomaly score S A (Anomaly Score)；
Step seven: from C CTC MTC with score less than the threshold is removed, if C CTC If the test is empty, returning to the step three, otherwise, inputting the candidate test into the C CTC Put into result set C RTC And returning to the step two.
Further, in the third step, the method for creating the metamorphic test case set includes: an initial Test input (STC) is generated by randomly selecting a partition from each input parameter, a subsequent Test input (FTC) is obtained according to the metamorphic relation, and the initial Test input (STC) and the Follow-up Test input (FTC) form an metamorphic Test input (Metamorphic Test Case, MTC) and the like.
Further, in the fourth step, the distance D is calculated sf The formula of (2) is:where n is the number of input parameters and i is the number of input parameters.
Further, in the sixth step, an abnormality score S is calculated A The formula of (2) is:wherein x is MTC, using C RTC Constructing an isolated forest as a training set, h (x) is x at the height of the isolated tree, E (h (x)) is the average of h (x), and C (n) is C RTC The average height of the isolated tree when the test input is equal to n is used for normalizing h (x), S A The closer to 1 means the greater degree of abnormality.
The invention has the following positive effects:
(1) Supporting metamorphic test, automatically generating test cases until all input domain partitions are at least covered by one test case;
(2) Supporting variability metrics for metamorphic test cases using a distance D of STC from FTC sf The difference degree of single MTC is represented, and considering that most binary MR in reality only contains one FTC, the calculation cost is small, and only two test cases participate in calculation, so the dimension influence is small;
(3) Supporting the difference measurement of the test cases relative to the result set, and measuring the candidate metamorphic test cases relative to the result set C by adopting the abnormal score of the isolated forest algorithm RTC The method has the advantages that the calculation complexity is O (n), the linear time complexity is realized, the speed is high when the high-dimensional data is processed, the memory consumption is low, and the method is commonly used for network attack detection, financial transaction fraud detection and the like;
(4) Calculating candidate test input and result set C without influence of input parameter dimension RTC When the difference is generated, a distance or density-based method is not used, so that the difference is not influenced by dimension;
(5) The probability of finding defects is improved, through two rounds of difference measurement, test inputs with obvious differences are automatically selected on the basis of a random value method, and the probability of finding defects is higher according to a defect distribution model;
(6) In view of cost, the number of test cases, especially the metamorphic test, is expected to be reduced as much as possible on the premise of keeping the error detection capability, so that the result set is very likely to be the small data set, and research shows that the isolated forest algorithm has better detection effect on the small data set.
Drawings
FIG. 1 is a schematic diagram of three types of software defect modes in software engineering practice;
FIG. 2 is a flow chart of an embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples.
Referring to fig. 2, taking a sine function sin as an example, the present implementation includes the following steps,
(1) Initializing: assume that the field [0,2 pi ] is to be input]Divided into [0,1 ]]、(1,π]、(π,5]、(5,2π]Assuming that no executed test cases exist, randomly generating a batch of input C according to an input domain RTC ＝Random(0,2π,10)＝{π/3,2,3,...,5π/3}；
(2) Check if all partitions of all parameters have been covered: if all the partitions of all the parameters at least comprise one test case, stopping the test if the detection result is true, otherwise, executing the next step;
(3) Selecting uncovered partitions, and randomly generating an metamorphic test case set: at [0,1]Interval random generation of 10 initial test inputs C STC Random (0, 1, 10) = {0.1,0.2,..1 }, periodically available metamorphic relationship MR from sin 1 ：x 2 ＝x 1 +2π，sin(x 2 )＝sin(x 1 ) Generating a subsequent test input C FTC ＝{C stc +2pi } = { 0.1+2pi, 0.2+2pi,.. MTC = { (0.1, 0.1+2pi), (0.2, 0.2+2pi),..;
(4) Calculating the distance between the initial input and the subsequent input: calculation C MTC Distance between STC and FTC of each MTC, D sf (MTC 1 )＝√(0.1-(0.1+2π)) 2 =2pi, other MTC, and so on;
(5) Taking the MTC group with the largest distance as a candidate test input set C CTC Assuming that the number of elements is 3, then from C MTC Middle fetch D sf Maximum 3 MTCs form C CTC ＝{MTC 1 ,MTC 2 ,MTC 3 }；
(6) Computing each MTC relative to the result set C using an isolated forest algorithm RTC Is of anomaly score S A Let S A (MTC 1 )＝0.98,S A (MTC 2 )＝0.50,S A (MTC 1 )＝0.43；
(7) From C CTC MTC with a score less than the threshold was removed, assuming an abnormal thresholding of 0.85, C after MTC removal CTC ＝{MTC 1 }，C CTC Not empty, input candidate test C CTC Put into result set C RTC Returning to the step 2, otherwise returning to the step 3.
The embodiment can embody the difference between the initial input and a plurality of subsequent inputs in the metamorphic test case; evaluating the difference between the candidate test case and the executed test case set by using an isolated forest anomaly detection algorithm; the calculation complexity is reduced to linear complexity, and the engineering practicability is achieved.
Various modifications and variations of the present invention may occur to those skilled in the art, and, if such modifications and variations are within the scope of the claims and their equivalents, they are also within the scope of the patent of the present invention.
What is not described in detail in the specification is prior art known to those skilled in the art.
Claims (1)
1. A self-adaptive random generation method for a test case of metamorphic test is characterized by comprising the following steps:
step one: initializing: the tested program includes several input parameters, each parameter has different value ranges, firstly according to the design document of the tested program, analyzing input domain, identifying equivalence class, according to equivalence class dividing input domain into n non-intersecting regions, for example, the input X = { X1, X2, …, xn }, the input domain of X1 is [0,100 }]Divided into [0, 20), [20, 50), [50,100 ]]Three regions, other input parameters and so on, initializing a Result Test Case, RTC C RTC Copy to executed test cases if they existC RTC Otherwise, randomly generating a batch of input C according to the input domain RTC ；
Step two: check if all partitions of all parameters have been covered: stopping the generation process if all partitions of all input parameters of the tested program are covered by at least one test input, and outputting C RTC Otherwise, executing the next step;
step three: selecting uncovered partitions, and randomly generating an metamorphic test case set: randomly generating k MTCs and establishing an MTC set C MTC ；
Step four: calculating the distance between the initial input and the subsequent input: calculation C MTC Distance D between STC and FTC of each MTC sf ；
Step five: taking a group of MTC with the largest distance as a candidate test input set Candidate Test Case and CTC CTC ；
Step six: computing each MTC relative to the result set C using an isolated forest algorithm RTC Abnormal Score S of Anomaly Score A ；
Step seven: from C CTC MTC with score less than the threshold is removed, if C CTC If the test is empty, returning to the step three, otherwise, inputting the candidate test into the C CTC Put into result set C RTC Returning to the second step;
in the third step, the method for establishing the candidate metamorphic test case set comprises the following steps: randomly selecting a partition from each input parameter to generate an initial Test input Source Test Case and STC, obtaining a Follow-up Test input Follow-up Test Case and FTC according to the metamorphic relation, forming an metamorphic Test input Metamorphic Test Case, MTC and the like;
in the fourth step, the distance D is calculated sf The formula of (2) is:wherein n is the number of input parameters, i is the number of input parameters;
in the sixth step, an anomaly score S is calculated A The formula of (2) is:wherein x is MTC, using C RTC Constructing an isolated forest as a training set, h (x) is x at the height of the isolated tree, E (h (x)) is the average of h (x), and C (n) is C RTC The average height of the isolated tree when the test input is equal to n is used for normalizing h (x), S A The closer to 1 means the greater degree of abnormality.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994979.9A CN113778865B (en) | 2021-08-27 | 2021-08-27 | Test case self-adaptive random generation method for metamorphic test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994979.9A CN113778865B (en) | 2021-08-27 | 2021-08-27 | Test case self-adaptive random generation method for metamorphic test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113778865A CN113778865A (en) | 2021-12-10 |
| CN113778865B true CN113778865B (en) | 2023-07-18 |
Family
ID=78839832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110994979.9A Active CN113778865B (en) | 2021-08-27 | 2021-08-27 | Test case self-adaptive random generation method for metamorphic test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113778865B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117056203B (en) * | 2023-07-11 | 2024-04-09 | 南华大学 | Numerical expression type metamorphic relation selection method based on complexity |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572462A (en) * | 2014-12-31 | 2015-04-29 | 中国人民解放军理工大学 | Method for generating metamorphic test case based on adaptive random strategy |
| CN104598381A (en) * | 2015-01-26 | 2015-05-06 | 中国人民解放军理工大学 | Positioning method for failure test instance in metamorphic testing |
| CN110823226A (en) * | 2019-10-30 | 2020-02-21 | 北京航空航天大学 | Unmanned aerial vehicle intelligent route planning test method based on metamorphic test technology |
| CN111124896A (en) * | 2019-12-06 | 2020-05-08 | 北京京航计算通讯研究所 | Metamorphic test system for primary and secondary peak ratio calculation algorithm |
| CN111830935A (en) * | 2020-07-23 | 2020-10-27 | 深圳慕智科技有限公司 | Metamorphic test method for automatic driving system |
-
2021
- 2021-08-27 CN CN202110994979.9A patent/CN113778865B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572462A (en) * | 2014-12-31 | 2015-04-29 | 中国人民解放军理工大学 | Method for generating metamorphic test case based on adaptive random strategy |
| CN104598381A (en) * | 2015-01-26 | 2015-05-06 | 中国人民解放军理工大学 | Positioning method for failure test instance in metamorphic testing |
| CN110823226A (en) * | 2019-10-30 | 2020-02-21 | 北京航空航天大学 | Unmanned aerial vehicle intelligent route planning test method based on metamorphic test technology |
| CN111124896A (en) * | 2019-12-06 | 2020-05-08 | 北京京航计算通讯研究所 | Metamorphic test system for primary and secondary peak ratio calculation algorithm |
| CN111830935A (en) * | 2020-07-23 | 2020-10-27 | 深圳慕智科技有限公司 | Metamorphic test method for automatic driving system |
Non-Patent Citations (1)
| Title |
|---|
| 基于程序路径分析的有效蜕变测试;董国伟;聂长海;徐宝文;;计算机学报(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113778865A (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107168842B (en) | Self-adaptive sequential fault diagnosis method based on PMC model | |
| Yao et al. | Genetic algorithm-based test data generation for multiple paths via individual sharing | |
| CN113778865B (en) | Test case self-adaptive random generation method for metamorphic test | |
| Grbac et al. | Stability of software defect prediction in relation to levels of data imbalance | |
| Milroy et al. | Towards characterizing the variability of statistically consistent Community Earth System Model simulations | |
| Williams | Learning to live with sampling variability: Expected replicability in partial correlation networks. | |
| Ding et al. | A framework for ensuring the quality of a big data service | |
| CN115859805A (en) | Self-adaptive sequential test design method and device based on mixed point adding criterion | |
| Pesteh et al. | A new interior point solver with generalized correntropy for multiple gross error suppression in state estimation | |
| Grentzelos et al. | A comparative study of methods to handle outliers in multivariate data analysis | |
| Pietsch et al. | Memory capacity and intelligence: Novel techniques for evaluating rival models of a fundamental information-processing mechanism | |
| Rönkkö et al. | Construct validity in partial least squares path modeling | |
| CN116611003A (en) | Transformer fault diagnosis method, device and medium | |
| CN109815108A (en) | A kind of combined test set of uses case priorization sort method and system based on weight | |
| CN116150687A (en) | Fluid pipeline leakage identification method based on multi-classification G-WLSTSVM model | |
| Huang et al. | Improving random test sets using a locally spreading approach | |
| Villanueva et al. | clustcurv: An r package for determining groups in multiple curves | |
| US11515995B2 (en) | Efficient computation of univariate statistical moments for side channel vulnerability evaluation | |
| Wang et al. | Fault detection capabilities of combinatorial testing and random testing for Boolean-specifications | |
| CN109726401B (en) | Patent combination generation method and system | |
| Hook et al. | Mutation sensitivity testing | |
| Kartha et al. | Why are you weird? infusing interpretability in isolation forest for anomaly detection | |
| CN111859799A (en) | Method and device for evaluating data accuracy based on complex electromechanical system coupling relation model | |
| Li | A simulation evaluation of backward elimination and stepwise variable selection in regression analysis | |
| TW202001246A (en) | Method for detecting drug resistance of microorganism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |