JP2017041197A - Test data generation device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate test data with high comprehensiveness under the consideration of a type of a symbol variable.SOLUTION: A symbolic execution part 11 is configured to extract a path condition by performing symbolic execution using a symbol variable to a test object program, and to generate test data fulfilling the extracted path condition, and a determination part 12 is configured to, on the basis of the path condition for which the test data fulfilling the path condition are not generated among the extracted path conditions, determine a changed type of the symbol variable included in the path condition, and a driver stub generation part 13 is configured to generate a driver and stub for performing symbolic execution to generate the test data again for the path condition for which the test data fulfilling the path condition are not generated by using the symbol variable whose type has been changed.SELECTED DRAWING: Figure 1

Description

  The disclosed technology relates to a test data generation device, a test data generation method, and a test data generation program.

  Symbolic that extracts the logical constraints that should be satisfied by the symbol variable as a path condition when the program is executed without passing the symbol (symbol variable), and the conditional variable is passed, without embodying the variable value in the program under test Execution technology exists. In symbolic execution, a driver that calls a test target program and gives an argument to a function in the test target program, and a stub that is a dummy implementation of a function called from the test target and returns a value when called. When the argument given from the driver and the value returned from the stub are symbol variables, test data covering the path of the test target program can be generated.

  Here, the program does not operate correctly if an appropriate type is not defined for a variable included in the program. The same applies to the driver and stub used for the symbolic execution described above, and the driver needs to give an appropriate type value as an argument, and the stub needs to return an appropriate type value.

  As a technique for generating test data for a program, a technique for generating description data to be input to a constraint solver from information on conditions for causing a program to execute a specific processing path has been proposed as a technique considering variable types. In this method, a variable for comparison corresponding to a numeric type variable is defined, and for the comparison condition constraint between a numeric type variable and a string type variable, a comparison variable is used instead of the numeric type variable. Set the constraints used.

  A method for generating a first test case by symbolic execution using a symbol variable including a control variable indicating the state of an object type variable input to a program to be tested has been proposed. In this method, a condition to be satisfied by the control variable is omitted from the path condition of the first test case, a second test case is generated by adding the condition to be satisfied by the object type variable as a state, and the second test is synonymous. Extract the third test case without the case.

JP 2013-254412 A JP 2014-6643 A

  However, for example, in a program written in a dynamically typed language, or in a program in which a default variable type is not given or an incorrect variable type is given, the type is different from the expected type. May be entered. In such a case, the conventional technology cannot generate test data.

  An object of the disclosed technique is to generate test data with high completeness in consideration of a symbol variable type.

  As one aspect, the disclosed technology includes a generation unit that extracts a path condition by symbolic execution using a symbol variable for a test target program and generates test data that satisfies the extracted path condition. . In addition, the disclosed technique is based on a path condition in which test data that satisfies the path condition is not generated among the path conditions extracted by the generation unit, and the symbol variable included in the path condition is changed. A determination unit for determining the type is provided. Further, the disclosed technique uses the symbol variable whose type has been changed by the determining unit to generate test data again for a path condition for which test data that satisfies the path condition has not been generated. The control part which controls is provided.

  The disclosed technology has an effect that, as one aspect, test data with high completeness can be generated in consideration of the type of the symbol variable.

It is a functional block diagram which shows schematic structure of the test data generation apparatus which concerns on this embodiment. It is a figure which shows an example of the program by which the comment regarding a variable type was described. It is a figure which shows an example of the driver which defined the type of the variable based on the comment, and the driver which considered null. It is a figure which shows an example of the program containing the conditional sentence which is not performed by type mismatch. It is a functional block diagram which shows schematic structure of the computer which functions as a test data generation apparatus which concerns on this embodiment. It is a flowchart which shows an example of the test data generation process in this embodiment. It is a figure which shows an example of a test object program. It is a figure which shows the production | generation result of an initial driver and test data. It is a figure which shows the production | generation result of the driver and test data after the type change of a symbol variable. It is a figure which shows the production | generation result of the driver and test data after the type change of a symbol variable.

  First, before describing the details of the embodiment, the types of symbol variables in the symbolic execution technique will be described.

  In order for the program to work properly, it is necessary that the variable has an appropriate type defined. At the time of symbolic execution, an appropriate type must be defined for the argument given by the driver and the value returned from the stub. Therefore, it is possible to obtain a variable type from the contents described in a comment (annotation) in the source code in the test target program and generate a driver and a stub that can give a variable in which an appropriate type is defined. Conceivable. For example, it is read from the comment part (part indicated by the broken line P in FIG. 2) of the test target program 131A as shown in FIG. 2 that the variables s1 and s2 are character strings (String type). A driver 122A as shown in FIG.

  However, in the test target program 131A shown in FIG. 2, it is necessary to consider the case where the variables s1 and s2 are null, but the types of the variables s1 and s2 given from the driver 122A are character strings. Therefore, it is not possible to generate test data considering that the variables s1 and s2 are null.

  For example, a symbol variable (* _isNull) that is true when the variables s1 and s2 are null may be defined as in the driver 122B illustrated in the lower part of FIG. In the driver 122B, s? _IsNull is true if true, s if false? It is defined that _body is substituted. Thereby, it is possible to generate test data including a case where the symbol variable is null.

  However, for example, in a program written in a dynamically typed language such as Java Script (registered trademark), the type of a variable is determined when the program is executed. In addition, in the early stage of program design, the type of input value may not be determined. For this reason, it is difficult to generate a driver and a stub that give a value in which an appropriate type is defined using a comment in the test target program 131A as shown in FIG. For example, in the test target program 131B shown in FIG. 4, the conditional statement indicated by the broken line Q can be executed only when the variable s2 is a numerical value. However, when the above-described driver 122A is used, since the type of the variable s2 is set as a character string, this conditional statement cannot be executed, and test data that passes this conditional statement is generated. I can't. The driver 122B also defines the symbol variable type in consideration of the null value, but test data can be generated when a value of a type completely different from the expected type is input. Therefore, there will be a path where the test cannot be executed.

  Therefore, in this embodiment, test data with high completeness is generated by changing the type of the symbol variable included in the path condition in which test data is not generated.

  Hereinafter, an exemplary embodiment related to the disclosed technology will be described in detail with reference to the drawings.

  As shown in FIG. 1, the test data generation apparatus 10 according to the present embodiment accepts a test target program 31 as an input, and generates and outputs test data 35 by symbolic execution.

  As shown in FIG. 1, the test data generation device 10 functionally includes a symbolic execution unit 11, a determination unit 12, and a driver / stub generation unit 13. The symbolic execution unit 11 is an example of a generation unit of the disclosed technology, and the driver / stub generation unit 13 is an example of a control unit of the disclosed technology.

  The symbolic execution unit 11 performs symbolic execution on the test target program 31 using the driver and stub generated by the driver / stub generation unit 13, extracts the path condition, and satisfies the extracted path condition Generate data. Further, the symbolic execution unit 11 notifies the determination unit 12 of a path condition for which test data could not be generated (hereinafter referred to as “uncovered path condition”).

  The determination unit 12 determines the changed type of the symbol variable included in the uncovered path condition based on the uncovered path condition. The possible types of symbol variables are predetermined as a symbol variable type set 15. In the present embodiment, in order to simplify the description, a case where there are three types of symbol variables, that is, a character string (String type), a numerical value (Number type), and null will be described.

  Specifically, the determination unit 12 specifies a symbol variable included in an uncovered path condition as a symbol variable whose type is to be changed. Then, the determination unit 12 determines the type of a constant other than the symbol variable included in the uncovered path condition as the changed type of the specified symbol variable. For example, when the uncovered path condition is “s1 == 1 (numerical value)”, the determination unit 12 identifies “s1” as the symbol variable whose type is to be changed, and determines the changed type as “numeric value”. To do.

  In addition, when the test data cannot be generated even after the symbol variable type is changed, the determination unit 12 selects one type from the symbol variable type set 15 and repeats the change of the symbol variable type. As the changed type of the selected symbol variable. For example, for the types included in the symbol variable type set 15, the determination unit 12 selects an unselected type according to a predetermined order such as, for example, character string → numerical value → null, and changes the specified symbol variable Later types can be determined. In addition, when a plurality of symbol variables are specified from uncovered path conditions, the determination unit 12 sequentially changes the combination of the types of the symbol variables. For example, when the specified symbol variables are two, s1 and s2, (character string, character string) → (character string, numeric value) → (character string, null) → (numeric value) as (s1 type, s2 type) , Character string) →... → (null, null). Then, the determination unit 12 can select a combination of unselected types according to this order, and determine the type after the change of the identified symbol variable.

  The determination unit 12 notifies the driver / stub generation unit 13 of the symbol variable whose type is to be changed and the changed type.

  The driver / stub generation unit 13 analyzes the test target program 31 and generates a driver and a stub used for symbolic execution of the test target program 31. The driver / stub generation unit 13 generates a driver and a stub in which the symbol variable type is changed based on the notification from the determination unit 12. The driver / stub generation unit 13 delivers the generated driver and stub to the symbolic execution unit 11.

  The test data generation device 10 can be realized by, for example, a computer 40 shown in FIG. The computer 40 includes a CPU 41, a memory 42 as a temporary storage area, and a nonvolatile storage unit 43. The computer 40 is connected to an input / output device 44 such as a display device and an input device, a read / write (R / W) unit 45 that controls reading and writing of data with respect to the recording medium 49, and a network such as the Internet. Network I / F 46. The CPU 41, the memory 42, the storage unit 43, the input / output device 44, the R / W unit 45, and the network I / F 46 are connected to each other via a bus 47.

  The storage unit 43 can be realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, or the like. A test data generation program 50 for causing the computer 40 to function as the test data generation device 10 is stored in the storage unit 43 as a storage medium.

  The CPU 41 reads the test data generation program 50 from the storage unit 43 and expands it in the memory 42, and sequentially executes the processes included in the test data generation program 50. The test data generation program 50 includes a symbolic execution process 51, a determination process 52, and a driver stub generation process 53. The CPU 41 operates as the symbolic execution unit 11 illustrated in FIG. 1 by executing the symbolic execution process 51. Further, the CPU 41 operates as the determination unit 12 illustrated in FIG. 1 by executing the determination process 52. The CPU 41 operates as the driver / stub generation unit 13 illustrated in FIG. 1 by executing the driver / stub generation process 53. As a result, the computer 40 that has executed the test data generation program 50 functions as the test data generation apparatus 10.

  The function realized by the test data generation program 50 can be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit).

  Next, the operation of the test data generation apparatus 10 according to this embodiment will be described. When the test target program 31 is input to the test data generation apparatus 10, the test data generation process shown in FIG. Here, it is assumed that a test target program 31 as shown in FIG. 7 is input. In the test target program 31 shown in FIG. 7, a comment that the types of the variables s1 and s2 are character strings (String type) is described (broken line R in FIG. 7).

  In step S <b> 11 of the test data generation process illustrated in FIG. 6, the driver / stub generation unit 13 generates an initial driver and stub d (0) from the test target program 31. Here, based on the comments of the test target program 31, the driver / stub generator 13 defines each of the variables s1 and s2 as a symbol variable of a character string (String type) as shown in the upper part of FIG. A driver 22A is generated. Note that the illustration of the stub is omitted. Then, the driver / stub generation unit 13 delivers the generated initial driver / stub d (0) to the symbolic execution unit 11. The symbolic execution unit 11 performs symbolic execution on the test target program 31 using the initial driver stub d (0), extracts a path condition, and generates test data that satisfies the extracted path condition. .

  The lower part of FIG. 8 shows a test data generation result 23A by symbolic execution using the driver 22A. As shown in the generation result 23A, four path conditions (Nos. 1 to 4) are extracted from the test target program 31. At this stage, since the type of each of the symbol variables s1 and s2 is a character string, the symbolic execution unit 11 performs No. 2 and no. Test data satisfying each of the three path conditions can be generated. However, no. The path condition of “s2 == 1 (numerical value)” of 1 is a type mismatch because the type of the symbol variable s2 at this stage is a character string, and the symbolic execution unit 11 performs test data that satisfies the path condition. Cannot be generated. Similarly, no. The path condition of “s1 == null” in FIG. 4 is also a type mismatch because the type of the current symbol variable s1 is a character string, and the symbolic execution unit 11 generates test data that satisfies the path condition. I can't. That is, no. 1 and no. Each of the four path conditions becomes an “uncovered path condition”. The symbolic execution unit 11 is No. 1 and no. The determination unit 12 is notified that each of the four path conditions is an uncovered path condition.

  If the test data generation result 23A shown in FIG. 8 is associated with the test target program 31 shown in FIG. 2 and no. 3 shows that test data for executing the test of part 3 has been generated. On the other hand, no. 1 and no. 4 indicates that test data for executing the test of part 4 has not been generated.

  Next, in step S <b> 12, among the uncovered path conditions notified from the symbolic execution section 11, whether there is an uncovered path condition that has not been subjected to symbol variable type change processing, which will be described later. Determine whether or not. If there is an uncovered path condition in which the symbol variable type change process has not been performed, the process proceeds to step S13.

  In step S13, the determination unit 12 selects one of the uncovered path conditions that have not been subjected to the symbol variable type change process as the path condition to be processed. Here, no. Assume that one pass condition is selected. Next, in step S14, the determination unit 12 sets a variable i indicating the number of symbol variable type changes to 0, and then in step S15, the determination unit 12 increments the variable i by one.

  Next, in step S16, the determination unit 12 identifies a symbol variable included in the processing target path condition as a symbol variable whose type is to be changed. Here, no. “S2” included in one path condition is specified as a symbol variable whose type is to be changed. Then, the determination unit 12 sets the type “numerical value (Number type)” of “1”, which is a constant other than the symbol variable “s2” included in the path condition to be processed, as the type after the change of the symbol variable “s2”. decide. The determining unit 12 notifies the driver / stub generating unit 13 of the symbol variable whose type is to be changed and the determined changed type.

  Next, in step S <b> 17, the driver stub generation unit 13 generates a driver stub d (i) in which the symbol variable type is changed based on the notification from the determination unit 12. Note that d (i) represents a driver stub whose symbol variable type has been changed i times. When j is the number of symbol variables and k is the number of types of symbols that can be taken by the symbol variable (the number of types included in the symbol variable type set 15), i is i = 0, 1,. 1 (N = j × k).

  Here, based on the notification from the determination unit 12, the driver / stub generation unit 13 changes the type of the symbol variable s2 from a character string (String type) to a numerical value (Number type) as shown in the upper part of FIG. The changed driver 22B is generated. Then, the driver / stub generation unit 13 delivers the generated driver 22B to the symbolic execution unit 11. The symbolic execution unit 11 uses the driver 22B to generate test data that satisfies the processing target path condition.

  The lower part of FIG. 9 shows a test data generation result 23B by the driver 22B. By changing the type of the symbol variable s2 to a numerical value, No. Test data satisfying one pass condition can be generated. As a result, the test program No. 31 shown in FIG. 2 and no. 3 in addition to No. 3 It shows that test data for executing the test of the part 1 is generated.

  Next, in step S18, the symbolic execution unit 11 determines whether or not test data that satisfies the path condition to be processed has been generated. If test data satisfying the processing target path condition is generated, the process returns to step S12. If not generated, the process proceeds to step S19. In step S19, the determination unit 12 determines whether i is smaller than N-1, that is, whether there is a further changeable type. If i <N−1, the process returns to step S15. If i ≧ N−1, the process returns to step S12. Here, the processing target No. Since the test data satisfying the path condition 1 has been generated, the process returns to step S12, and in step S13, no. 4 is selected as the next path condition to be processed.

  In step S16, the determination unit 12 determines whether the processing target No. 4 is specified as a symbol variable whose type is to be changed, and “null” is determined as a type after the change of the symbol variable “s1” from the path condition to be processed. The determining unit 12 notifies the driver / stub generating unit 13 of the symbol variable whose type is to be changed and the determined changed type.

  Next, in step S17, based on the notification from the determination unit 12, the driver / stub generation unit 13 changes the type of the symbol variable s1 from a character string (String type) to null as shown in the upper part of FIG. The changed driver 22C is generated. Then, the driver / stub generation unit 13 delivers the generated driver 22C to the symbolic execution unit 11. The symbolic execution unit 11 uses the driver 22C to generate test data that satisfies the processing target path condition.

  The lower part of FIG. 10 shows a test data generation result 23C by the driver 22C. By changing the type of the symbol variable s1 to null, no. Test data that satisfies the four pass conditions can be generated. As a result, the test program No. 31 shown in FIG. 1-No. 3 in addition to No. 3 4 shows that test data for executing the test of part 4 has been generated. That is, it indicates that test data covering all the paths included in the test target program 31 has been generated.

  If a negative determination is made in step S12, the process proceeds to step S20, the symbolic execution unit 11 outputs the generated test data 35, and the test data generation process ends.

  In the above-described test data generation process, the case where an attempt is made to change each type of symbol variable until the test data is generated for the path condition to be processed has been described, but the present invention is not limited to this. The number of executions of the mold changing process may be limited to a predetermined number.

  As described above, according to the test data generation device according to the present embodiment, when the test data that satisfies the path condition is not generated, the type of the symbol variable included in the path condition is changed to the path condition. Change to the type of other constants included in. If the test data is still not generated, the type of the symbol variable is sequentially changed to a type selected from the types that the symbol variable can take. This makes it possible to generate test data with high completeness compared to the case where test data is generated by symbolic execution using a symbol variable of a predetermined type.

  This is particularly effective when a program to be tested is a program that is described in a dynamically typed language and whose variable type is changed. In addition, even if the target variable is not given the default variable type or the program is given the wrong variable type, the symbol variable type is changed from the uncovered path condition. High test data can be generated. Therefore, it is possible to perform a test by symbolic execution from an early stage of program design in which the variable type is not fixed.

  Further, in the case of a driver that considers a null value, such as the driver 122B shown in FIG. 3, the type of the symbol variable is defined in consideration of whether or not it is null in any case. For this reason, there is a problem that the memory required for symbolic execution, the code of the driver and the stub become large. According to the present embodiment, symbolic execution for each time does not determine whether or not the type of one symbol variable is null, and does not consider a plurality of types according to the determination result. Necessary memory and code size of driver and stub can be suppressed.

  In the above description, the test data generation program 50 is stored (installed) in the storage unit 43 in advance. However, the present invention is not limited to this. The program according to the disclosed technology can be provided in a form recorded on a recording medium such as a CD-ROM, a DVD-ROM, or a USB memory.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
For the test target program, a symbolic execution using a symbol variable extracts a path condition and generates a test data that satisfies the extracted path condition;
A determination unit that determines a changed type of a symbol variable included in the path condition based on a path condition for which test data that satisfies the path condition is not generated among the path conditions extracted by the generation unit; ,
Using a symbol variable whose type has been changed by the determination unit, a control unit that controls the generation unit to generate test data again for a path condition for which test data that satisfies the path condition is not generated;
Test data generation device including

(Appendix 2)
The test data generation according to appendix 1, wherein the determination unit determines a type of a constant other than a symbol variable included in a path condition for which test data that satisfies the path condition is not generated as a type after the symbol variable is changed. apparatus.

(Appendix 3)
The test data according to supplementary note 2, wherein the determination unit sequentially selects an unselected type from a plurality of types until a test data that satisfies a path condition is generated, and determines the type after the change of the symbol variable. Generator.

(Appendix 4)
The control unit performs symbolic execution on the test target program, and generates a driver and a stub that give the symbol variable of the type determined by the determination unit at the time of the symbolic execution. The test data generation device according to any one of the above.

(Appendix 5)
On the computer,
For the program to be tested, symbolic execution using symbol variables extracts path conditions and generates test data that satisfies the extracted path conditions.
Of the extracted path conditions, based on a path condition for which test data that satisfies the path condition is not generated, a type of the symbol variable included in the path condition is changed, and
A test data generation method that executes processing including generating test data again for a path condition for which test data that satisfies the path condition is not generated using a symbol variable whose type has been changed.

(Appendix 6)
In the process of determining the changed type of the symbol variable, a constant type other than the symbol variable included in the path condition for which test data that satisfies the path condition is not generated is used as the changed type of the symbol variable. The test data generation method according to appendix 5, which is determined.

(Appendix 7)
In the process of determining the changed type of the symbol variable, unselected types are sequentially selected from a plurality of types until test data that satisfies a path condition is generated, and the changed type of the symbol variable is selected. The test data generation method according to appendix 6, which is determined as follows.

(Appendix 8)
The process of generating test data again includes performing symbolic execution on the test target program and generating a driver and a stub that give a symbol variable of the determined type at the time of the symbolic execution. The test data generation method according to any one of?

(Appendix 9)
On the computer,
For the program to be tested, symbolic execution using symbol variables extracts path conditions and generates test data that satisfies the extracted path conditions.
Of the extracted path conditions, based on a path condition for which test data that satisfies the path condition is not generated, a type of the symbol variable included in the path condition is changed, and
A test data generation program for executing a process including generating test data again for a path condition for which test data that satisfies the path condition is not generated using a symbol variable whose type has been changed.

(Appendix 10)
In the process of determining the changed type of the symbol variable, a constant type other than the symbol variable included in the path condition for which test data that satisfies the path condition is not generated is used as the changed type of the symbol variable. The test data generation program according to appendix 9, which is determined.

(Appendix 11)
In the process of determining the changed type of the symbol variable, unselected types are sequentially selected from a plurality of types until test data that satisfies a path condition is generated, and the changed type of the symbol variable is selected. The test data generation program according to appendix 10, which is determined as

(Appendix 12)
The process of generating test data again includes performing symbolic execution on the test target program and generating a driver and a stub that give a symbol variable of the determined type at the time of the symbolic execution. The test data generation program according to any one of to 11.

(Appendix 13)
On the computer,
For the program to be tested, symbolic execution using symbol variables extracts path conditions and generates test data that satisfies the extracted path conditions.
Of the extracted path conditions, based on a path condition for which test data that satisfies the path condition is not generated, a type of the symbol variable included in the path condition is changed, and
A storage medium storing a test data generation program for executing processing including generating test data again for a path condition for which test data that satisfies the path condition is not generated using a symbol variable whose type has been changed .

DESCRIPTION OF SYMBOLS 10 Test data generation apparatus 11 Symbolic execution part 12 Determination part 13 Driver stub generation part 15 Symbol variable type | mold set 22A, 22B, 22C Driver 31 Test object program 35 Test data 40 Computer 41 CPU
42 Memory 43 Storage 49 Recording medium 50 Test data generation program

Claims (6)

For the test target program, a symbolic execution using a symbol variable extracts a path condition and generates a test data that satisfies the extracted path condition;
A determination unit that determines a changed type of a symbol variable included in the path condition based on a path condition for which test data that satisfies the path condition is not generated among the path conditions extracted by the generation unit; ,
Using a symbol variable whose type has been changed by the determination unit, a control unit that controls the generation unit to generate test data again for a path condition for which test data that satisfies the path condition is not generated;
Test data generation device including   2. The test data according to claim 1, wherein the determination unit determines a constant type other than the symbol variable included in the path condition for which the test data satisfying the path condition is not generated as the changed type of the symbol variable. Generator.   3. The test according to claim 2, wherein the determination unit sequentially selects an unselected type from a plurality of types until a test data that satisfies a path condition is generated, and determines the type after the change of the symbol variable. Data generator.   The control unit performs symbolic execution on the test target program, and generates a driver and a stub that give a symbol variable of a type determined by the determination unit at the time of the symbolic execution. Item 4. The test data generation device according to any one of Items 3 to 3. On the computer,
For the program to be tested, symbolic execution using symbol variables extracts path conditions and generates test data that satisfies the extracted path conditions.
Of the extracted path conditions, based on a path condition for which test data that satisfies the path condition is not generated, a type of the symbol variable included in the path condition is changed, and
A test data generation method that executes processing including generating test data again for a path condition for which test data that satisfies the path condition is not generated using a symbol variable whose type has been changed. On the computer,
For the program to be tested, symbolic execution using symbol variables extracts path conditions and generates test data that satisfies the extracted path conditions.
Of the extracted path conditions, based on a path condition for which test data that satisfies the path condition is not generated, a type of the symbol variable included in the path condition is changed, and
A test data generation program for executing a process including generating test data again for a path condition for which test data that satisfies the path condition is not generated using a symbol variable whose type has been changed.

Images