CN102508770A - Test case suite amplification method based on self-adaption random testing of predicate - Google Patents
Test case suite amplification method based on self-adaption random testing of predicate Download PDFInfo
- Publication number
- CN102508770A CN102508770A CN2011103069433A CN201110306943A CN102508770A CN 102508770 A CN102508770 A CN 102508770A CN 2011103069433 A CN2011103069433 A CN 2011103069433A CN 201110306943 A CN201110306943 A CN 201110306943A CN 102508770 A CN102508770 A CN 102508770A
- Authority
- CN
- China
- Prior art keywords
- predicate
- test
- test case
- use cases
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 202
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000003321 amplification Effects 0.000 title claims abstract description 42
- 238000003199 nucleic acid amplification method Methods 0.000 title claims abstract description 42
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000005516 engineering process Methods 0.000 claims abstract description 18
- 230000003044 adaptive effect Effects 0.000 claims abstract description 5
- 238000000605 extraction Methods 0.000 claims abstract description 5
- 230000003068 static effect Effects 0.000 claims description 13
- 230000004048 modification Effects 0.000 claims description 12
- 239000000284 extract Substances 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 235000000332 black box Nutrition 0.000 claims description 6
- 239000012141 concentrate Substances 0.000 claims description 5
- 238000002474 experimental method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 3
- 238000002386 leaching Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000003416 augmentation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000013522 software testing Methods 0.000 description 1
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention provides a test case suite amplification method based on self-adaption random testing of a predicate. In the invention, a branch conditional expression in a process is called a branch; an expression backtracking to an input field from a branch condition is called the predicate and is used for regressing testing; and a white box testing technology for correcting and analyzing software and a black box testing technology for ART (Adaptive Random Testing) are combined, first, predicate extraction is carried out, then test case suit amplification is carried out by the ART under the guidance of predicate distance. In the invention, due to the combination of the white box testing technology for correcting and analyzing software and the black box testing technology for the ART, the effective amplification of the test case suite with low cost is achieved.
Description
Technical field
The invention belongs to the software testing technology field; Especially regression test technical field in the software test; Be used to carry out the amplification of test use cases, realize new version software is more fully tested, be a kind of test use cases amplification method based on the test of predicate self-adapting random.
Background technology
Because factors such as function modification, Performance tuning, software reconfiguration and wrong reparations, software need be revised evolution.Regression test is software modification to be detected to guarantee it does not introduce new mistake.Existing regression test research is divided into three major types usually: test case is selected (Test Case Selection), test case reparation (Test Case Repairing), and test use cases amplification (Test Suite Augmentation).The main task that test case is selected is selected from original test case exactly, and according to target difference can be divided three classes: regression test selection, test use cases yojan and test case priority.The modification of software possibly make that original test case is unavailable; The target of test case reparation is that it is repaired into available test case.The test use cases amplification is then carried out more comprehensively regression test through newly-increased test case.
The regression test research of past 20 years mainly concentrates on test case and selects.Yet develop during some of test use in software-oriented, reuse part in addition all test case can not satisfy the testing requirement that software develops and brings fully.For example, original test case often can not be tested newly-increased module.Thereby need new test case to satisfy testing requirement.The test use cases amplification is extracted new testing requirement through the software modification analysis, and obtains new test case through fc-specific test FC use-case generation method.The main challenge of test use cases amplification is how software modification analysis and test use-case to be generated combination, to reduce testing cost and to improve the regression tested precision.Current test use cases amplification is mainly based on test case generation techniques such as symbol execution, heuristic searches.The white-box testing technology is fit to combine with the software modification analysis, often is subject to too high cost but carry out the test case generation with it.
Summary of the invention
The problem that the present invention will solve is: existing test case amplification method can not satisfy the requirement of the amplification of test use cases in the regression test to testing cost and measuring accuracy, and the new test case amplification method of needs satisfy these demands.
Technical scheme of the present invention is: a kind of test use cases amplification method based on the test of predicate self-adapting random; Branch condition expression formula in the program is called branch, and the expression formula that is dateed back input domain by branch condition is called predicate, is used for regression test; White-box testing technology in conjunction with software modification is analyzed is technological with the Black-box Testing of self-adapting random test ART; At first carry out predicate and extract, carry out the test use cases amplification through predicate apart from guiding ART then, may further comprise the steps:
1) uses the system of selection of Dejavu test case, concentrate from former test case and select to constitute " test use cases of selection " through revising the test case of part;
2) static analysis regression tested redaction program, in the extraction procedure the predicate that might be extracted;
3) distance between the Jaccard distance calculation test case of employing predicate sequential value: the predicate that extracts is formed a predicate set, and predicate quantity is designated as
N All , the quantity of equal values is designated as in the predicate result set
N Same Use test use-case then
tThe static predicate of carrying out obtains a predicate results set R; Calculate the predicate distance between test case, the predicate distance between test case through predicate results set R
D(
t 1 , t 2 ) adopt the Jaccard distance calculation, as follows:
With gained predicate distance definition is the distance between test case;
4) carry out adaptive testing use-case generative process based on predicate; Generate new test use cases; Detailed process is: on input domain, generate n test case at random, the test case that calculate to generate respectively and selected the predicate distance between each test case of test use cases, and with the gained minor increment as the test case of this generation and selected the distance between the test use cases; Then with in this n test case with selected test use cases apart from that of maximum as the test case that increases newly; Repeat this process m time, whole process stops, and m is for needing the test case quantity of amplification; When perhaps in the time of setting, not increasing test case new, whole process stops.
As preferably, in the step 4), on input domain, generate n test case at random, n=10.
Step 2) predicate in is extracted as: static state is recalled the expression formula of being extracted from program branches, and is expressed as the expression formula of input domain variable and program constant, and its concrete form is following:
ExpressionΘConstant
Θ representes the comparison symbol, and Expression representes an expression formula that comprises variable, and Constant representes constant, and other form for predicate is shown as this form through map table; For the variable that contains non-input parameter among the Expression, be expressed as the form of input parameter through stipulations, the predicate form after the stipulations is following:
Expression?only?with?argumentsΘConstant
When predicate extracts:
1) ignores the branch of containing global variable;
2) ignore the predicate that contains built-in function and third party's function call.
Characteristics of the present invention are: 1, propose the predicate extractive technique; 2, the definition of predicate distance is proposed; 3, adopt the predicate distance to carry out the self-adapting random test; 4, the combination test case is selected technology and is carried out the test use cases amplification based on the adaptive testing technology of predicate.Show that through experiment the present invention can cover through reach higher branch than low-cost.
The white-box testing technology that the inventive method combines software modification to analyze is technological with the Black-box Testing of ART; Through introducing new distance metric mode--predicate distance; Guiding ART carries out the test use cases amplification, thereby reaches effective amplification of the test use cases of lower cost.The experience of test use cases amplification shows as far as possible program information and test use cases information of utilizing original version more, and the test case that can help to increase reaches better test effect.To software modification, we at first adopt white box technology, cover on the basis with software modification information analyzing existing branch, and location needs the branch condition of covering to gather, and branch condition static state is dateed back input domain obtain new predicate expression formula.Through introducing new distance metric mode--predicate distance, guiding ART Black-box Testing technology is carried out effective test use cases amplification then.The combination of Program Static Analysis technology and Black-box testing Cases generation technique can effectively reduce new test case and generate cost, improves the measuring accuracy of newly-increased test case.
Description of drawings
Fig. 1 is a process flow diagram of the present invention.
The exemplary plot that Fig. 2 extracts for predicate of the present invention.
Fig. 3 utilizes the inventive method to 15 new test cases of 0.5% test use cases amplification.
Fig. 4 utilizes the inventive method to 10 new test cases of 1% test use cases amplification.
Embodiment
Like Fig. 1, the present invention combines the white-box testing technology of software modification analysis and the Black-box Testing technology of self-adapting random test to increase for the test use cases in the regression test, makes it through reach higher branch's coverage rate than low-cost, and step is following:
1) use the system of selection of Dejavu test case, concentrate from former test case and select to constitute " having selected test use cases " through revising the test case of part, other test case then is removed among original test use cases T;
2) static analysis regression tested redaction program, in the extraction procedure the predicate that might be extracted; Branch condition expression formula in the program is called " branch ", and the expression formula that is dateed back input domain by branch condition is called " predicate ".Predicate is extracted as: static state is recalled the expression formula of being extracted from program branches, and is expressed as the expression formula of input domain variable and program constant, and its concrete form is following:
ExpressionΘConstant
Θ representes the comparison symbol, and Expression representes an expression formula that comprises variable, and Constant representes constant, and other form for predicate is shown as this form through map table; For the variable that contains non-input parameter among the Expression, be expressed as the form of input parameter through stipulations, the predicate form after the stipulations is following:
Expression?only?with?argumentsΘConstant
When predicate extracts:
(1),, therefore ignores the branch of containing global variable because the global variable value could confirm that when program run the global variable value is difficult to static definite if contain global variable in the predicate;
(2),, therefore ignore this type of predicate because this class function is difficult to static analysis if contain built-in function and third party's function calls in the predicate.What pay special attention to is, though more predicate can be expressed more accurate " predicate distance ", technology of the present invention does not also require that the equal static state of all branches recalls into predicate.
Predicate leaching process example such as Fig. 2, in this example, example procedure has 3 different predicates, and (P1, P2 P3), introduce leaching process: P1 below and meet the object form that extracts predicate, so directly output; P2 has a paths to lead to program entry, exports behind the elimination intermediate variable; P3 has two paths to lead to program entry, so the predicate stipulations are P3A and P3B, extract and exports after intermediate variable is eliminated in the back.The predicate sequence of example procedure is as shown in table 1.
Table 1: the predicate abstraction sequence of example procedure
3) distance between the Jaccard distance calculation test case of employing predicate sequential value: the predicate that extracts is formed a predicate set, and predicate quantity is designated as
N All , the quantity of equal values is designated as in the predicate result set
N Same The static predicate of carrying out of use test use-case t obtains a predicate results set R then; Calculate the predicate distance between test case, the predicate distance between test case through predicate results set R
D(
t 1 , t 2 ) adopt the Jaccard distance calculation, as follows:
Gained predicate distance is the distance between test case;
4) carry out adaptive testing use-case generative process, generate new test use cases based on predicate.Detailed process is: on input domain, generate n test case at random; Experience shows: the n=10 effect is best; Through calculating the test case that generates and having selected the predicate distance between each test case of test use cases; And with the gained minor increment as this test case and selected the distance between the test use cases, then with in this n test case with selected test use cases apart from that of maximum as the test case that increases newly.Repeat this process m time, the test case quantity of m for needing to increase, the new test case that perhaps in the time of setting, do not increase, whole process stops.
Below enforcement of the present invention will be described through concrete embodiment.
Through verifying the validity of this test use cases amplification technique.In order to prove validity of the present invention, we will contrast the result of random test RT.Therefore, at experimental section the test use cases amplification technique of the present invention and RT has been used in the amplification of original test use cases respectively, relatively the branch of the two covers rate of growth.
1, experimental subjects
When choosing experimental subjects, our target is to select the experimental arrangement of classics, and this program is input as numeric type.Therefore, the experimental subjects of selection is the Tcas program of Siemens experiment external member, and its advantage is to comprise perfect test use cases and relevant test script, and is used by Many researchers, belongs to classical experimental arrangement.
2, assessment level
The present invention below is called based on P-ART test case amplification method, and target is the branch of covering tested program as much as possible, and branch's coverage rate of therefore selecting test use cases is as examination criteria.Test use cases T for branch's coverage rate PT computing method be:
Concrete detection based on the method for the validity of P-ART test case amplification method is; On the test use cases of selecting, carry out the amplification of P-ART and the amplification that random test generates test case (Random Testing), relatively the test use cases after the amplification is with respect to the rate of growth of amplification Pretesting set of uses case.The test use cases of setting through generating based on P-ART amplification back is T
P-ART, the test use cases that generates through Random Testing amplification mode is T
RTT then
P-ARTAnd T
RTThe difference Growth Rate Calculation following:
Experiment has used Codecover to carry out pitching pile when the branch of calculating test use cases covers, accomplishes the calculating of branch's coverage rate.
3, experiment is implemented
Based on Java Platform Implementation whole algorithm process, change the Tcas program into the Java language version by the C language version.Test case selects to have used Sofya, and it provides the structure of code pitching pile with control flow graph (CFG), and the RTS module (Dejavu) that is used for the test case selection is provided.Aspect the predicate extraction, in conversion Tcas program language version, we adopt the form of target predicate to convert the C language version into the Java version, the predicate sequence in therefore can direct output writer.
4, interpretation of result
Because the test use cases quantity that the Tcas program carries is bigger, but program itself is smaller, the control flow structure is single relatively, and this just causes a part of test case seldom just can reach very high even branch's covering completely.For fear of such problem, guarantee the accuracy of experimental data simultaneously, we concentrate from former test case respectively and select 0.5% and 1% at random, and each selection is repeated to do 15 experiments.30 groups experimental result is shown in Fig. 3 and 4; P-ART refers to
among the figure, and RT refers to
).
The Comprehensive Experiment data see that under the equal conditions, the present invention is superior to random test (RT) use-case generation method; Secondly, under the not high situation of coverage rate of former test use cases, the present invention is based on P-ART test case amplification method and can reach good effect.
Claims (3)
1. test use cases amplification method based on predicate self-adapting random test; Branch condition expression formula in the program is called branch, and the expression formula that is dateed back input domain by branch condition is called predicate, it is characterized in that being used for regression test; White-box testing technology in conjunction with software modification is analyzed is technological with the Black-box Testing of self-adapting random test ART; At first carry out predicate and extract, carry out the test use cases amplification through predicate apart from guiding ART then, may further comprise the steps:
1) uses the system of selection of Dejavu test case, concentrate from former test case and select to have selected test use cases through revising the test case of part, constituting;
2) static analysis regression tested redaction program, all predicates that can be extracted in the extraction procedure;
3) distance between the Jaccard distance calculation test case of employing predicate sequential value: the predicate that extracts is formed a predicate set, and predicate quantity is designated as N
All, the quantity of equal values is designated as N in the predicate result set
SameThe static predicate of carrying out of use test use-case t obtains a predicate results set R then; Calculate the predicate distance between test case, the predicate distance B (t between test case through predicate results set R
1, t
2) adopt the Jaccard distance calculation, as follows:
D(t
1，t
2)＝1-N
same/N
all
With gained predicate distance definition is the distance between test case;
4) carry out adaptive testing use-case generative process based on predicate; Generate new test use cases; Detailed process is: on input domain, generate n test case at random, the test case that calculate to generate respectively and selected the predicate distance between each test case of test use cases, and with the gained minor increment as the test case of this generation and selected the distance between the test use cases; Then with in this n test case with selected test use cases apart from that of maximum as the test case that increases newly; Repeat this process m time, whole process stops, and m is for needing the test case quantity of amplification; When perhaps in the time of setting, not increasing test case new, whole process stops.
2. a kind of test use cases amplification method based on the test of predicate self-adapting random according to claim 1 is characterized in that in the step 4), on input domain, generates n test case at random, n=10.
3. a kind of test use cases amplification method according to claim 1 and 2 based on the test of predicate self-adapting random; It is characterized in that step 2) in predicate be extracted as: staticly from program branches recall the expression formula of being extracted; And be expressed as the expression formula of input domain variable and program constant, its concrete form is following:
ExpressionΘConstant
Θ representes the comparison symbol, and Expression representes an expression formula that comprises variable, and Constant representes constant, and other form for predicate is shown as this form through map table; For the variable that contains non-input parameter among the Expression, be expressed as the form of input parameter through stipulations, the predicate form after the stipulations is following:
Expression?only?with?argumentsΘConstant
When predicate extracts:
1) ignores the branch of containing global variable; 2) ignore the predicate that contains built-in function and third party's function call.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110306943.3A CN102508770B (en) | 2011-10-10 | 2011-10-10 | Test case suite amplification method based on self-adaption random testing of predicate |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110306943.3A CN102508770B (en) | 2011-10-10 | 2011-10-10 | Test case suite amplification method based on self-adaption random testing of predicate |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102508770A true CN102508770A (en) | 2012-06-20 |
| CN102508770B CN102508770B (en) | 2015-02-11 |
Family
ID=46220862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110306943.3A Expired - Fee Related CN102508770B (en) | 2011-10-10 | 2011-10-10 | Test case suite amplification method based on self-adaption random testing of predicate |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102508770B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279422A (en) * | 2013-06-17 | 2013-09-04 | 东南大学 | Self-adaptive random test method based on rejecting area |
| CN108959056A (en) * | 2017-05-17 | 2018-12-07 | 富士通株式会社 | Method for generating test case and Test cases technology device |
| CN110688310A (en) * | 2019-09-20 | 2020-01-14 | 中国人民解放军63921部队 | Software test case distance calculation method based on attraction |
| CN111813669A (en) * | 2020-07-04 | 2020-10-23 | 毛澄映 | Adaptive random test case generation method based on multi-target group intelligence |
| CN114416598A (en) * | 2022-03-28 | 2022-04-29 | 广州番禺职业技术学院 | Crowdsourcing test amplification method based on test slice |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102103539A (en) * | 2011-03-11 | 2011-06-22 | 天津大学 | Z-specification-based test case generating method |
-
2011
- 2011-10-10 CN CN201110306943.3A patent/CN102508770B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102103539A (en) * | 2011-03-11 | 2011-06-22 | 天津大学 | Z-specification-based test case generating method |
Non-Patent Citations (4)
| Title |
|---|
| JIANG B, ET AL.: "Adaptive random test case prioritization", 《AUTOMATED SOFTWARE ENGINEERING, 2009. ASE"09. 24TH IEEE/ACM INTERNATIONAL CONFERENCE ON. IEEE》 * |
| ROTHERMEL G，ET AL.: "Prioritizing test cases for regression testing", 《SOFTWARE ENGINEERING, IEEE TRANSACTIONS ON》 * |
| ROTHERMEL G.ET AL.: "A safe, efficient regression test selection technique", 《ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY (TOSEM)》 * |
| TAI K C.: "Predicate-based test generation for computer programs", 《SOFTWARE ENGINEERING, 1993. PROCEEDINGS.15TH INTERNATIONAL CONFERENCE ON. IEEE》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279422A (en) * | 2013-06-17 | 2013-09-04 | 东南大学 | Self-adaptive random test method based on rejecting area |
| CN103279422B (en) * | 2013-06-17 | 2015-09-09 | 东南大学 | A kind of method that self-adapting random based on repulsive area is tested |
| CN108959056A (en) * | 2017-05-17 | 2018-12-07 | 富士通株式会社 | Method for generating test case and Test cases technology device |
| CN108959056B (en) * | 2017-05-17 | 2021-11-16 | 富士通株式会社 | Test case generation method and test case generation device |
| CN110688310A (en) * | 2019-09-20 | 2020-01-14 | 中国人民解放军63921部队 | Software test case distance calculation method based on attraction |
| CN111813669A (en) * | 2020-07-04 | 2020-10-23 | 毛澄映 | Adaptive random test case generation method based on multi-target group intelligence |
| CN111813669B (en) * | 2020-07-04 | 2023-10-13 | 毛澄映 | Adaptive random test case generation method based on multi-target group intelligence |
| CN114416598A (en) * | 2022-03-28 | 2022-04-29 | 广州番禺职业技术学院 | Crowdsourcing test amplification method based on test slice |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102508770B (en) | 2015-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102722440B (en) | A kind of debugging apparatus of embedded system executable code, adjustment method and embedded system | |
| Nguyen et al. | Studying the impact of dependency network measures on software quality | |
| US10108536B2 (en) | Integrated automated test case generation for safety-critical software | |
| CN102508770B (en) | Test case suite amplification method based on self-adaption random testing of predicate | |
| US8402444B2 (en) | Program analysis through predicate abstraction and refinement | |
| WO2018121531A1 (en) | Method and apparatus for generating test case script | |
| CN109117364A (en) | A kind of object-oriented method for generating test case and system | |
| US20130091495A1 (en) | Feedback-directed random class unit test generation using symbolic execution | |
| US9965252B2 (en) | Method and system for generating stateflow models from software requirements | |
| CN109933515B (en) | Regression test case set optimization method and automatic optimization device | |
| KR20180129623A (en) | Apparatus for statically analyzing assembly code including assoxiated multi files | |
| CN109739759A (en) | A method of verifying Black-box testing Cases design integrity | |
| CN105718368A (en) | Software testing data amplification method | |
| CN103309805A (en) | Automatic selection method for test target in object-oriented software under xUnit framework | |
| CN105045715A (en) | Programming mode and mode matching based bug clustering method | |
| Huang et al. | Prioritizing variable-strength covering array | |
| KR20110023124A (en) | Apparatus and method for verification for automotive ecu software | |
| CN104572474A (en) | Dynamic slicing based lightweight error locating implementation method | |
| CN104536882A (en) | Error locating method based on frequent sub-graph mining | |
| CN114138669A (en) | Software automatic testing method based on function level selection symbolized mixed execution | |
| Kukolj et al. | Selection and prioritization of test cases by combining white-box and black-box testing methods | |
| JPWO2019142266A1 (en) | Test case generation device, test case generation method, and test case generation program | |
| CN108132799B (en) | Inter-process static program analysis information extraction method, device and equipment | |
| CN104536880A (en) | GUI program testing case augmentation method based on symbolic execution | |
| Srivastava et al. | Cause effect graph to decision table generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150211 Termination date: 20161010 |