CN108647146B - Test case generation method for judging combination coverage based on correction condition - Google Patents

Abstract

The invention discloses a test case generation method based on correction condition judgment combination coverage, which comprises the steps of source code preprocessing analysis, control tree generation method analysis, control tree optimization based on correction condition judgment combination coverage and test case generation. The method can obviously improve the coverage rate of the software correction condition judgment combination compared with the currently used automatic test tool, and meanwhile, the method also relatively improves the sentence coverage, judgment coverage and simple condition coverage criteria, has few test cases and high coverage rate, and has obvious advantages in the aspect of improving the coverage rate of the software test cases.

Description

Test case generation method for judging combination coverage based on correction condition

Technical Field

The invention relates to the field of software testing, in particular to a test case generation method for judging combined coverage based on correction conditions.

Background

Software testing is a process for executing program error searching, is an important stage of a software life cycle, and is a key step for ensuring software quality. The cost of software test accounts for more than 50% of the total development cost, and the effective test data generation method can generate high-quality test cases, thereby reducing the generation scale of the test cases and the software development cost.

The automatic test case generation mode can be different according to the adopted method and different research objects. In the prior art, a test case is generated based on a genetic algorithm, and a natural selection principle and a search algorithm of a natural genetic mechanism are used, so that unique advantages and high efficiency can be displayed when high complexity problems such as large space, nonlinearity, multiple peaks and the like are solved, but the actual application effect is not optimistic when some implementation problems are met; some scholars use a traditional search algorithm, namely a hill climbing method to generate test cases, and the method has the advantages that the local optimal solution is good for finding, so that the method can produce a good effect when solving the one-dimensional predicate, but the method is difficult to realize when solving the diversity of the predicate and achieving a high search success rate; kangoye Sekou proposes three methods for automatically generating test cases based on Modified Condition Decision Combination (MCDC), but does not consider the possible dependency relationship between two or more standard decisions, so that the number of test cases is large, and the coverage rate is not high.

Disclosure of Invention

The invention aims to provide a test case generation method for judging combination coverage based on correction conditions, so as to reduce the number of test cases and improve the software test coverage.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a test case generation method based on correction condition judgment combination coverage comprises the following steps:

step 1, preprocessing and analyzing source codes: extracting all variables in the program and converting all variables involved in the control conditions into the relation between the input variables and the constants by analyzing the source program;

and 2, analyzing a generation method of the control tree: firstly, generating an intermediate file of a source program by using a source code after preprocessing analysis through a GCC compiler, wherein the intermediate file stores statement block information and control condition information in the source program and the relationship among the statement block information and the control condition information; then, replacing the information in the intermediate file by using the line number information, then representing the replaced line number information by using character strings, storing the line number information into a list, storing all side information of the control tree into the list, and finally generating the control tree of a source program by processing the side information in the list;

the control tree is a triplet T ═ N, E, N₀) N represents all node sets in the control tree, each node set consists of a basic block and a judgment, if the judgment comprises a plurality of conditions, the plurality of conditions are required to be disassembled into a single condition, the first condition of a second line is represented by '2 _ 1', the line number in the program is used for representing the serial number of the node, and if the node is a basic block, the serial number with the minimum line number is used for representing the serial number; e represents all directed edge sets in the control tree; n is₀Representing an entry node unique to the program;

and 3, judging optimization of the combined coverage on the control tree based on the correction condition: firstly, finding out different combinations of a plurality of conditions, after the short circuit condition is eliminated, judging whether each combination meets the executable condition, namely whether the combination meets the actual condition, if a certain combination does not meet the actual condition, removing the combination, then finding out the combination meeting the coverage criterion of the correction condition judgment combination from the rest combinations, meeting all the conditions of the coverage criterion, and thus finding out a control tree based on the coverage criterion of the correction condition judgment combination;

the correction condition judgment combination coverage criterion is that the following conditions are simultaneously met: each inlet in the program enters at least once, and each outlet exits at least once; all possible outcomes of any decision in the routine occur at least once; all possible outcomes for each condition arbitrarily determined in the routine occur at least once; each single condition in the program can independently influence the result of the corresponding judgment, namely, the result of only one condition is ensured to be changed in the judgment, and the value of the judgment can be changed when the value results of other conditions are not changed;

step 4, generating a test case: combining the optimization process of the control tree, extracting all control condition information on a given path, and solving constraints through a solver to obtain a test case;

the path is a set of inequalities and equations from an entry node to a leaf node in the control tree, and is defined by PC ═ P_i,C_ij,B_ijDenotes, P_iRepresenting the ith path in the function to be tested; c_ijRepresents the jth condition on the ith path, and the conditions all represent a single condition; b is_ijThe return values of the jth condition on the ith path are represented by T, F and NG, wherein NG represents that the condition does not influence the truth of the judgment in the function to be tested, and if B in one path is executed_ijIf the condition is equal to F, inverting the condition and then solving; if B is present_ijNG indicates a value without taking this condition into consideration.

As a further preferred aspect of the present invention, step 1 comprises the following operations: for a module, firstly, input variables corresponding to the module are found and extracted, all assignment statements are found and extracted, self-increment and self-decrement operators are treated as special assignment statements, all control conditions are found according to extraction of keywords, the relation between each variable and the input variables is found according to the assignment statements, and then the variables in the control conditions are replaced and modified according to the execution sequence of the statements.

As a further preferable aspect of the present invention, the step 2 further comprises the following operations:

if the control node of a program contains a plurality of conditions, two possibilities of true and false exist for each single condition, when the conditions are the same, that is, when the left values of the arrow symbols shown in the list are the same, the character string with the small index value on the right side of the arrow symbol is the case that the condition value is true, and the character string with the large index value on the right side of the arrow symbol is the case that the condition value is false.

As a further preferable aspect of the present invention, the step 2 further comprises the steps of:

in the process of generating the control tree, when a loop condition is faced, a concept of z-path covering is adopted, wherein the concept of z-path covering is that three loop structures while, for and do-while all adopt a two-branch structure, two branches of while and for are respectively an execution loop body and an non-execution loop body, and two branches of do-while are respectively a twice-execution loop body and a once-execution loop body.

The invention has the following beneficial effects: compared with the currently used automatic test tool, the method can obviously improve the software MCDC coverage rate, relatively improves the statement coverage, judgment coverage and simple condition coverage criteria, has few test cases and high coverage rate, and has obvious advantages in improving the coverage rate of the software test cases.

Drawings

FIG. 1 is a flow chart of a test case generation method of the present invention;

FIG. 2 is an exemplary program segment according to a first embodiment of the present invention;

FIG. 3 is an exemplary pseudo code segment in accordance with a first embodiment of the present invention;

FIG. 4 is a control tree of an exemplary pseudo code segment in accordance with a first embodiment of the present invention;

FIG. 5 is an optimized control tree of an exemplary pseudo code segment in accordance with a first embodiment of the present invention;

FIG. 6 is an exemplary process of a second embodiment of the present invention;

FIG. 7 is an exemplary process of a third embodiment of the present invention;

FIG. 8 is a comparison of coverage of test cases generated by an automated test tool according to a third embodiment of the present invention;

FIG. 9 shows coverage of test cases according to a third embodiment of the present invention;

Detailed Description

The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

The first embodiment is as follows: the embodiment is a test case generation method for judging combination coverage based on correction conditions, which comprises the following steps:

step 1, preprocessing and analyzing source codes: extracting all variables in the program and converting all variables involved in the control conditions into the relation between the input variables and the constants by analyzing the source program;

and 2, analyzing a generation method of the control tree: firstly, generating an intermediate file of a source program by using a source code after preprocessing analysis through a GCC compiler, wherein the intermediate file stores statement block information and control condition information in the source program and the relationship among the statement block information and the control condition information; then, replacing the information in the intermediate file by using the line number information, then representing the replaced line number information by using character strings, storing the line number information into a list, storing all side information of the control tree into the list, and finally generating the control tree of a source program by processing the side information in the list;

the control tree is a triplet T ═ N, E, N₀) N represents all node sets in the control tree, each node set consists of a basic block and a judgment, if a plurality of conditions are contained in the judgment, the plurality of conditions need to be disassembled into a single condition, and the judgment is carried out in a mode of' 21' indicates the first condition of the second row, the number of the node is indicated by the row number in the program, and if the node is a basic block, the number with the smallest row number is indicated; e represents all directed edge sets in the control tree; n is₀Representing an entry node unique to the program;

and 3, judging optimization of the combined coverage on the control tree based on the correction condition: firstly, finding out different combinations of a plurality of conditions, after the short circuit condition is eliminated, judging whether each combination meets the executable condition, namely whether the combination meets the actual condition, if a certain combination does not meet the actual condition, removing the combination, then finding out the combination meeting the coverage criterion of the correction condition judgment combination from the rest combinations, meeting all the conditions of the coverage criterion, and thus finding out a control tree based on the coverage criterion of the correction condition judgment combination;

the correction condition judgment combination coverage criterion is that the following conditions are simultaneously met: each inlet in the program enters at least once, and each outlet exits at least once; all possible outcomes of any decision in the routine occur at least once; all possible outcomes for each condition arbitrarily determined in the routine occur at least once; each single condition in the program can independently influence the result of the corresponding judgment, namely, the result of only one condition is ensured to be changed in the judgment, and the value of the judgment can be changed when the value results of other conditions are not changed;

step 4, generating a test case: combining the optimization process of the control tree, extracting all control condition information on a given path, and solving constraints through a solver to obtain a test case;

the path is a set of inequalities and equations from an entry node to a leaf node in the control tree, and is defined by PC ═ P_i,C_ij,B_ijDenotes, P_iRepresenting the ith path in the function to be tested; c_ijRepresents the jth condition on the ith path, and the conditions all represent a single condition; b is_ijThe return values of the jth condition on the ith path are represented by T, F and NG, wherein NG represents that the condition does not influence the judgment in the function to be testedTrue or false, if B in a certain path of execution_ijIf the condition is F, the condition needs to be inverted, and then solution is carried out; if B is present_ijIf NG, this indicates a value that does not need to be considered.

The second embodiment is as follows: the present embodiment is a limitation of the first embodiment, and the step 1 includes the following operations:

for a module, firstly, input variables corresponding to the module are found and extracted, all assignment statements are found and extracted, self-increment and self-decrement operators are treated as special assignment statements, all control conditions are found according to extraction of keywords, the relation between each variable and the input variables is found according to the assignment statements, and then the variables in the control conditions are replaced and modified according to the execution sequence of the statements.

The third concrete implementation mode: this embodiment is a limitation of the second embodiment, and the step 2 further includes the following operations:

if the control node of a program contains a plurality of conditions, two possibilities of true and false exist for each single condition, when the conditions are the same, that is, when the left values of the arrow symbols shown in the list are the same, the character string with the small index value on the right side of the arrow symbol is the case that the condition value is true, and the character string with the large index value on the right side of the arrow symbol is the case that the condition value is false.

The fourth concrete implementation mode: in this embodiment, the third embodiment is a limitation to the first embodiment, and the step 2 further includes the following operations:

in the process of generating the control tree, when a loop condition is faced, a concept of z-path covering is adopted, wherein the concept of z-path covering is that three loop structures while, for and do-while all adopt a two-branch structure, two branches of while and for are respectively an execution loop body and an non-execution loop body, and two branches of do-while are respectively a twice-execution loop body and a once-execution loop body.

The present invention is described in further detail below with reference to specific examples, which are provided for the purpose of illustration only and are not intended to be limiting.

The first embodiment is as follows: step 1, as shown in fig. 2 for a program segment, if a specified path is a path returning to 1, it needs to be satisfied that both of two determination conditions x >3 and x <6 on the path take true values, however, in an actual program execution process, before a second condition x <6 is executed, an operation x + +, which is performed on the value of x, changes an original expression, and finally, a correct constraint expression should be changed to x >3 and x +1< 6.

The above operation is mainly divided into 3 steps: and extracting the input variables, finding all assignment statements, and converting all variables in the control condition into the relationship between the input variables and the constants according to the relationship between the assignment statements. As shown in the above example, for a certain module, we need to first find an input variable x corresponding to the module, extract all assignment statements according to the execution sequence of the program, treat the self-increment and self-decrement operators as special assignment statements, such as x + +, which can be converted into x ═ x +1, finally find all control conditions according to the extraction of the keywords, find the relationship between each variable and the input variable according to the assignment statements, and further replace and modify the variables in the control conditions according to the execution sequence of the statements, such as the second condition x <6 in the above example. However, no processing is needed for the first control condition x >3, and since the variable x is not changed when the condition is executed, no replacement is needed, and after the operation of step 1, more accurate constraint conditions can be extracted, thereby increasing the accuracy of generating the test case.

Step 2, as for the program shown in fig. 3, the format of the control node if (a & & B | | C) in the second row stored in the list is [2_1- >2_2,2_1- >2_3,2_2- >4,2_2- >2_3,2_3- >4,2_3- >12], there are two possibilities of true and false for each single condition, and when the conditions are the same, that is, when the left side value of '- >' shown in the list is the same, the character string with the small index value on the right side of '- >' is the case where the condition value is true, and the character string with the large index value on the right side of '- >' is the case where the condition value is false, the generated control tree is shown in fig. 4.

According to the following method, information from the root node to the leaf nodes in the control tree can be stored, which facilitates the implementation of the optimization process of the control tree. The method is shown in table 1.

TABLE 1

Step 3, in the example pseudo code segment of fig. 3, a multi-condition decision if (a & & B | | C) is included, there are many different combination cases for the result of each boolean value of the decision, theoretically there are eight theoretical combination cases for the decision including 3 conditions, but when the program is actually executed, a certain condition may mask the values of other conditions, for example, the decision a & & B | C, and when the condition a is F, the value of a & & B is F, only the value of the condition C needs to be considered, that is, the true or false of the whole decision can be known, taking the decision a & & B | C as an example, and after considering this, the condition combination case is reduced from the combination of theoretical 8 to 5, as shown in table 2, where 'NG' indicates that the true or false of the decision value is irrelevant to this variable.

TABLE 2

Thus, for the above determination with multiple conditions, after the optimization of the method proposed in step 3, the combination of the above 5 cases is reduced to 4 cases, as shown in table 3:

TABLE 3

Since 'NG' indicates that the truth of the judgment value is irrelevant to the variable, the truth of the whole judgment cannot be influenced no matter the value of the variable is true or false, and the four formed test cases can sufficiently meet the MCDC coverage criterion. All possible outcomes of any decision in the procedure occur at least once because t1 and t4 satisfy the condition of MCDC coverage criteria; t1, t3, and t4 satisfy the condition of the MCDC override criterion-all possible outcomes of each condition arbitrarily decided in the program occur at least once; t1, t3, t4 and t5 all satisfy the condition of the MCDC coverage criterion, each single condition in the program can independently affect the result of the corresponding judgment, that is, it is ensured that only the result of one condition is changed in the judgment, and when the value taking results of other conditions are not changed, the value of the judgment can be changed. And the condition (r) is the general requirement of the current programming rule. The MCDC combination pair (t1, t5) may demonstrate the independent influence of condition a, the MCDC combination pair (t3, t4) may demonstrate the independent influence of condition C, the MCDC combination pair (t3, t5) may demonstrate the independent influence of condition B, and condition (r) is also considered to be a unique reason for satisfying the MCDC coverage criterion. The control tree structure at this time is shown in fig. 5.

Therefore, the number of test cases can be obviously reduced by the method, and the algorithm finds out the combination condition meeting the MCDC coverage criterion on the basis of the multi-condition combination, so that the condition meeting the MCDC coverage criterion is required to meet the MCDC coverage.

However, it should be noted that, from the multi-condition coverage for excluding the short-circuit condition, it is first necessary to determine whether each combination meets the executable condition, i.e. whether it meets the actual condition, if a certain combination does not meet the actual condition, the combination should be removed, and the test case meeting the MCDC coverage criterion is selected from the remaining combinations. Such as x>7&&x+y<＝12||x<7&&y>5, judging, when the judged value is true, considering the short circuit condition of the compiler, wherein the combination condition of the two paths P1 and P2 is as follows: (B)₁₁＝T,B₁₂＝F,B₁₃＝T,B₁₄＝F；②B₂₁＝T,B₂₂＝F,B₂₃＝T,B₂₄T. For C₁₁And C₁₃、C₂₁And C₂₃It is not practical to have T, i.e. the two combinations just taken are not true at all, so it is necessary to remove the two combinations from the multi-conditional combination.

The solution to finding different combination cases from the multi-conditional combination is to translate the combination problem into a graph path traversal problem, as shown in table 4.

TABLE 4

After the combination algorithm, the short circuit condition is removed, then each combination condition is judged, and further a combination meeting the MCDC coverage criterion is found, namely all conditions meeting the coverage criterion are met, and then the control tree based on the MCDC coverage criterion is found.

Step 4, combining with the optimization process of the control tree, after a certain path is given, we only need to extract all the control condition information on the path, that is, the path constraint information PC ═ P_i,C_ij,B_ijAnd solving the constraint through a solver to obtain the test case.

Example two: since the research content is a method for generating test cases based on the MCDC coverage criterion, it is meaningful to research decisions containing multiple conditions, and if each decision contains only one condition, the decision must satisfy the MCDC coverage criterion, so the example program is shown in FIG. 6. For the procedure shown in fig. 6, the test case set and coverage distribution generated based on MCDC are shown in tables 5 and 6, respectively.

TABLE 5

TABLE 6

The result shows that no matter which coverage criterion is used for calculation, the coverage rate of the test case is obviously improved in the test case set generated based on the MCDC coverage mode, and the value taking result of each judgment and the value taking result of each condition in the judgment are considered when the test case is generated based on the MCDC coverage criterion, so that the successful coverage of elements which cannot be covered in the test case generated by only considering other coverage criteria is ensured.

Example three: example three is a comparison test, which finally tests the effectiveness of the method according to the coverage of the test case set by comparing with an automated test tool, parasoftett 9.2.1.26. The basic program code is a program that verifies triangles, as shown in FIG. 7.

The analysis was first performed using the automated test tool parasoft test9.2.1.26, the coverage results obtained from the set of test cases automatically generated by the tool are shown in fig. 8, and the intermediate results generated by the tool can result in the set of test cases shown in table 7.

TABLE 7

And (4) analyzing results: according to the combination of the single condition value taking result and the judgment value taking combination, the independent influence of the condition 1(a < b + c) can be verified for the 1 st to 3 rd use case numbers 1 and 3, and the independent influence of the condition 2(b < a + c) can be verified for the use case numbers 1 and 2; since the 4 th condition (| flag) is a single condition that must satisfy the MCDC coverage criterion, the set of test cases generated by the tool can only verify the independent influence of 3 conditions, thus yielding a coverage result as shown in fig. 6.

The condition value results on each path obtained by the method of the invention are combined to generate a test case set, as shown in table 8.

TABLE 8

And (4) analyzing results: according to the combination of the single condition value taking result and the judgment value taking combination, for the 1 st to 3 rd conditions, the independent influence of the condition 1(a < b + c) can be verified by the case number 1 and the case number 4, the independent influence of the condition 2(b < a + c) can be verified by the case number 1 and the case number 3, and the independent influence of the condition 3(c < a + b) can be verified by the case number 1 and the case number 2; for the 5 th and 6 th conditions, the use case No. 1 and the use case No. 6 can verify the independent influence of the condition 5(a ═ b); according to the combination of the value results of the 7 th to 9 th conditions, the independent influence of the condition 8 (b! ═ c) can be verified by the use case number 6 and the use case number 8, and the independent influence of the condition 9 (a! ═ c) can be verified by the use case number 6 and the use case number 7; therefore, in addition to the condition 6 and the condition 7, the test case set can realize the verification of the independent influence of each condition, so that after the test case set generated by the tool is replaced by the test case set, the obtained test coverage result is shown in fig. 9. Therefore, the results of comparing the coverage obtained after generating test cases using the automation tool parasoft test9.2.1.26 and using the test cases generated based on the MCDC criteria are shown in table 9.

Table 9 coverage comparison results

The results of the examples show that: the test case generated based on the MCDC criterion can obviously improve the software MCDC coverage rate, and meanwhile, the statement coverage, the judgment coverage and the simple condition coverage criterion are relatively improved, so that the test shows that the method has certain improvement in the aspect of improving the coverage rate.

Finally, it should be noted that: the above embodiments and examples are only for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments and examples, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments and examples can be modified, or some of the technical features can be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments and examples of the present invention.

Claims (4)

1. A test case generation method for judging combination coverage based on correction conditions is characterized by comprising the following steps:

step 1, preprocessing and analyzing source codes: extracting all variables in the program and converting all variables involved in the control conditions into the relation between the input variables and the constants by analyzing the source program;

and 2, analyzing a generation method of the control tree: firstly, generating an intermediate file of a source program by using a source code after preprocessing analysis through a GCC compiler, wherein the intermediate file stores statement block information and control condition information in the source program and the relationship among the statement block information and the control condition information; then, replacing the information in the intermediate file by using the line number information, then representing the replaced line number information by using character strings, and storing the line number information into a list, namely storing all the side information of the control tree into a list, and finally generating the control tree of a source program by processing the side information in the list;

the control tree is a triplet T ═ N, E, N₀) N represents all node sets in the control tree, each node set consists of a basic block and a judgment, if the judgment comprises a plurality of conditions, the plurality of conditions are disassembled into a single condition, the first condition of a second line is represented by '2 _ 1', the line number in the program represents the number of the node, and if the node is a basic block, the number with the minimum line number represents the node; e represents all directed edge sets in the control tree; n is₀Representing an entry node unique to the program;

and 3, judging optimization of the combined coverage on the control tree based on the correction condition: firstly, finding out different combinations of a plurality of conditions, after the short circuit condition is eliminated, judging whether each combination meets the executable condition, namely whether the combination meets the actual condition, if a certain combination does not meet the actual condition, removing the combination, then finding out the combination meeting the coverage criterion of the correction condition judgment combination from the rest combinations, meeting all the conditions of the coverage criterion, and thus finding out a control tree based on the coverage criterion of the correction condition judgment combination;

the correction condition judgment combination coverage criterion is that the following conditions are simultaneously met: each inlet in the program enters at least once, and each outlet exits at least once; all possible outcomes of any decision in the routine occur at least once; all possible outcomes for each condition arbitrarily determined in the routine occur at least once; each single condition in the program can independently influence the result of the corresponding judgment, namely, the result of only one condition is ensured to be changed in the judgment, and the value of the judgment can be changed when the value results of other conditions are not changed;

step 4, generating a test case: combining the optimization process of the control tree, extracting all control condition information on a given path, and solving constraints through a solver to obtain a test case;

the path is a set of inequalities and equations from an entry node to a leaf node in the control tree, and is defined by PC ═ P_i,C_ij,B_ijMeans for，P_iRepresenting the ith path in the function to be tested; c_ijRepresents the jth condition on the ith path, and the conditions all represent a single condition; b is_ijThe return values of the jth condition on the ith path are represented by T, F and NG, wherein NG represents that the condition does not influence the truth of the judgment in the function to be tested, and if B in one path is executed_ijIf the condition is equal to F, inverting the condition and then solving; if B is present_ijNG indicates a value without taking this condition into consideration.

2. The method according to claim 1, wherein the step 1 comprises the following operations:

for a module, firstly, input variables corresponding to the module are found and extracted, all assignment statements are found and extracted, self-increment and self-decrement operators are treated as special assignment statements, all control conditions are found according to extraction of keywords, the relation between each variable and the input variables is found according to the assignment statements, and then the variables in the control conditions are replaced and modified according to the execution sequence of the statements.

3. The method according to claim 2, wherein the step 2 further comprises the following operations:

if the control node of a program contains a plurality of conditions, two possibilities of true and false exist for each single condition, when the conditions are the same, that is, when the left values of the arrow symbols shown in the list are the same, the character string with the small index value on the right side of the arrow symbol is the case that the condition value is true, and the character string with the large index value on the right side of the arrow symbol is the case that the condition value is false.

4. The method according to claim 3, wherein the step 2 further comprises the following operations:

in the process of generating the control tree, when a loop condition is faced, a concept of z-path covering is adopted, wherein the concept of z-path covering is that three loop structures while, for and do-while all adopt a two-branch structure, two branches of while and for are respectively an execution loop body and an non-execution loop body, and two branches of do-while are respectively a twice-execution loop body and a once-execution loop body.

Images