JP5971116B2 - Test data generation method, program, and apparatus - Google Patents

Description

  The present invention relates to a technique for generating test data used for testing a program.

  Symbolic execution is a technique for executing a program by converting variables into symbols (that is, with symbols) without embodying variables in the program. Variables that are symbolized are called symbol variables. During symbolic execution, not a specific value but a condition to be satisfied by a symbol variable (hereinafter referred to as a path condition) is held in a memory or the like as a symbol variable value. When the symbolic execution is completed, a path condition is obtained for each path in the program. Therefore, test data for testing the program can be obtained by obtaining a specific value of the symbol variable that satisfies the path condition.

  For example, consider symbolic execution of a program having a problem as an absolute value acquisition function shown in FIG. FIG. 1 shows the code of a program to be symbolically executed and the control structure of the program. This program contains a variable X. When symbolic execution is performed on this program, test data X = −2 (pass condition is X <0), test data X = 1234 (pass condition is X ≧ 0 and X = 1234), and X = 3 (pass conditions are X ≧ 0 and X ≠ 1234).

  However, as described above, normal symbolic execution covers all paths in the program, and may execute uselessly. For example, if you do not want to include all paths in the program as a target range, but only need to target all instructions in the program, executing a symbolic execution will execute a path that does not contribute to improving instruction coverage. Will do. As a result, extra time is required to complete symbolic execution, and resources such as memory are wasted.

  There are conventional techniques that focus on such problems. However, in the prior art, since it is assumed that there are a plurality of symbol variables, the case where there is only one symbol variable is not considered. In some cases, even if this conventional technique is used, test data cannot be generated efficiently.

JP 2012-068869 A

  In one aspect, an object of the present invention is to provide a technique for efficiently generating exhaustive test data.

  The test data generation method according to the present invention includes a first process for performing symbolic execution while omitting a part of a target range of symbolic execution, and calculating a coverage rate for the symbolic execution, and a calculated coverage rate. The second process for determining whether or not the predetermined value has been exceeded, and if the calculated coverage rate does not exceed the predetermined value, at least a part of the omitted part of the range is set to be excluded. Then, a third process for executing the process after the first process again, and a value satisfying the path condition obtained by the symbolic execution when the calculated coverage rate exceeds a predetermined value are stored in the first data. 4th process stored in a part.

  Comprehensive test data can be generated efficiently.

FIG. 1 is a diagram for explaining symbolic execution. FIG. 2 is a functional block diagram of the symbolic execution device. FIG. 3 is a diagram showing a main processing flow. FIG. 4 is a diagram illustrating an example of the target program. FIG. 5 is a diagram illustrating an example of data stored in the target coverage storage unit. FIG. 6 is a diagram illustrating a processing flow of symbolic execution. FIG. 7 is a diagram illustrating an example of data stored in the second instruction data storage unit. FIG. 8 is a diagram illustrating an example of a call graph. FIG. 9 is a diagram illustrating a processing flow of symbolic execution. FIG. 10 is a diagram illustrating an example of data stored in the first function data storage unit. FIG. 11 is a diagram illustrating a process flow of the determination process. FIG. 12 is a diagram illustrating a processing flow of determination processing. FIG. 13 is a diagram illustrating an example of data stored in the first instruction data storage unit. FIG. 14 is a diagram illustrating an example of data stored in the second function data storage unit. FIG. 15 is a diagram for explaining a leaf node identification method as a reference. FIG. 16 is a diagram for explaining a coping method for recursive calls. FIG. 17 is a diagram for explaining path pruning by the processing of the present embodiment. FIG. 18 is a diagram illustrating a main processing flow when the range of path pruning is not controlled. FIG. 19 is a diagram illustrating a processing flow of symbolic execution when the range of path pruning is not controlled. FIG. 20 is a diagram illustrating another example of the target program. FIG. 21 is a diagram illustrating a result when the range of path pruning is not controlled. FIG. 22 is a diagram illustrating a result when the range of path pruning is not controlled. FIG. 23 is a functional block diagram of a computer.

  FIG. 2 shows a functional block diagram of the symbolic execution device 1 in the present embodiment. The symbolic execution device 1 includes an input unit 100, a target coverage storage unit 101, a target program storage unit 102, a range control unit 103, an execution engine 104 including an execution unit 112 and a generation unit 113, a determination unit 105, First instruction data storage unit 106, second instruction data storage unit 107, state data storage unit 108, first function data storage unit 109, second function data storage unit 110, call graph data storage unit 111, And a test data storage unit 114.

  The input unit 100 receives an input of the target program from the user and stores it in the target program storage unit 102. The input unit 100 receives an input of target coverage from the user and stores it in the target coverage storage unit 101.

  The range control unit 103 performs processing based on the list of functions that have undergone path pruning in the second function data storage unit 110 and the data of the call graph stored in the call graph data storage unit 111, and the processing result is displayed as the first function. Store in the data storage unit 109. In the present embodiment, “perform path pruning” means that the symbolic execution for the path is terminated halfway (that is, no more symbolic execution is performed for the path).

  The execution engine 104 is a module that performs symbolic execution, and is, for example, Java (registered trademark) PathFinder. The execution unit 112 performs symbolic execution on the program stored in the target program storage unit 102 and subjected to symbolic execution, and stores the executed instruction in the second instruction data storage unit 107. Further, the execution unit 112 manages data related to the execution state of the program (for example, the value of the program counter indicating the execution position, the contents of the call stack, the information about the symbol variable, and the path condition) in the state data storage unit 108. . In addition, the execution unit 112 stores the function that has performed the path pruning in the second function data storage unit 110. The execution unit 112 also generates call graph data and stores it in the call graph data storage unit 111. The generation unit 113 generates test data that satisfies a path condition obtained by symbolic execution, and stores the test data in the test data storage unit 114.

  The determination unit 105 determines whether there is an unexecuted instruction at the branch destination based on the list of executed instructions stored in the second instruction data storage unit 107 and the data stored in the state data storage unit 108. The process which determines is performed. In addition, the determination unit 105 stores the searched branch instruction in the first instruction data storage unit 106.

  Next, the operation of the symbolic execution device 1 will be described with reference to FIGS. First, the input unit 100 in the symbolic execution device 1 receives an input of a target program and target coverage to be subjected to symbolic execution from the user (FIG. 3: step S1). The input unit 100 stores the target program in the target program storage unit 102 and stores the target coverage in the target coverage storage unit 101. The target coverage is, for example, a target value for at least one of instruction coverage, condition coverage, and branch coverage.

  FIG. 4 shows an example of the target program. The target program shown in FIG. 4 is a program that calls the function “int chk2nd (s)” in the code representing the main process and calls the function “int strlen (char * s)” in the function “int chk2nd (s)”. . The number of instructions is 19.

  For example, as shown in FIG. 5, the target coverage is stored in the target coverage storage unit 101. A plurality of different types of target coverage may be stored (for example, a target value for instruction coverage and a target value for branch coverage may be stored).

  When the target program is stored in the target program storage unit 102, the execution unit 112 in the execution engine 104 increments an integer n (initial value is 0) for specifying a hierarchy in the call graph (step S3).

  The execution unit 112 initializes the execution state p by initializing data relating to the execution state stored in the state data storage unit 108 (step S5). The initialization is, for example, returning the value of the program counter to 1, clearing the contents of the call stack, clearing information about the state of symbol variables in the memory, and clearing the path condition.

  The execution unit 112 performs symbolic execution (null, p) (step S7). The symbolic execution (s, p) will be described with reference to FIGS. Note that the first character string in parentheses represents a list of executed instructions, and the second character string in parentheses is a character string representing an execution state. Here, (null, p) is to execute symbolic execution from the initial execution state p when the list of executed instructions (that is, the list in the second instruction data storage unit 107) is empty. Represents.

  First, the execution unit 112 reads one instruction from the target program stored in the target program storage unit 102 based on the program counter in the execution state p, and adds it to the second instruction data storage unit 107 (FIG. 6: S21). . The second instruction data storage unit 107 stores a list of executed instructions, for example, as shown in FIG.

  The execution unit 112 determines whether the read instruction is a function call instruction (step S23).

  If it is not a function call instruction (step S23: No route), the process proceeds to step S27. When it is a function call instruction (step S23: Yes route), the execution unit 112 adds the function to be called to the call graph specified based on the data stored in the call graph data storage unit 111 (step S25). ).

  FIG. 8 shows an example of a call graph. The example of FIG. 8 shows a call graph generated for the program shown in FIG. The call graph represents a function call relationship. In the following, each function in the call graph is called a node, and the most recently called function is called a leaf node. In the example of FIG. 8, the function “int string (char * s)” is a leaf node. In the call graph, the leaf node is a node in the lowest hierarchy.

  The execution unit 112 determines whether the read instruction is a branch related to a symbol variable (step S27). The branch related to the symbol variable is, for example, “if (string (s) <= 1)” with the row number 9 in FIG.

  If it is a branch related to the symbol variable (step S27: Yes route), the process proceeds to step S41 in FIG.

  Shifting to the description of FIG. 9, the execution unit 112 copies the execution state p to the execution state P, and specifies one unprocessed branch destination (execution state is P) in the branch (FIG. 9: Step S41). ). For example, when the branch is an if statement, there are Yes paths and No paths as branch destinations.

  The execution unit 112 determines whether the function being executed (that is, the function most recently called in the call graph) is not subject to path pruning (step S43). The determination in step S43 is performed based on whether or not the function being executed is included in the list of functions that are not subject to path pruning in the first function data storage unit 109. For example, data as shown in FIG. 10 is stored in the first function data storage unit 109.

  If the path is not subject to path pruning (step S43: Yes route), the execution of the symbolic execution (s, P) is not performed because the symbolic execution does not have to be terminated for the path (step S51). Here, (s, P) indicates that symbolic execution is performed for the execution state P when the list of executed instructions (that is, the list in the second instruction data storage unit 107) is in the state s. ing.

  On the other hand, if the function being executed is not excluded from path pruning (step S43: No route), the execution unit 112 requests the determination unit 105 to execute a determination process. In response, the determination unit 105 executes determination processing (s, P, null) (step S45). The determination process will be described with reference to FIGS. The first character string in the parentheses represents a list of instructions that have been executed, the second character string in the parentheses is a character string indicating an execution state, and the third character string in the parentheses is A character string representing a list of branch instructions that have been searched. Here, (s, P, null) is a list of instructions that have been executed (that is, a list in the second instruction data storage unit 107) that is already executed (ie, a list in the second instruction data storage unit 107) and that has already been searched (that is, In the state where the list in the first instruction data storage unit 106 is empty, the execution state P is determined.

  First, the determination unit 105 reads one instruction in the execution state p (FIG. 11: step S61).

  The determination unit 105 determines whether the read instruction is stored in the list of executed instructions in the second instruction data storage unit 107 (step S63). If not included (step S63: No route), the process proceeds to step S81 in FIG. Then, the determination unit 105 sets a value indicating that the instruction in the branch destination path is “unexecuted” as a return value (FIG. 12: step S81). Then, the process returns to the original process.

  On the other hand, when the read instruction is stored in the list of executed instructions in the second instruction data storage unit 107 (step S63: Yes route), it is determined whether the read instruction is “return” (step S65). ). If it is “return” (step S65: Yes route), it is considered that all the instructions of the path have been executed, so the processing shifts to step S83 in FIG. Then, the determination unit 105 sets a value indicating that the instruction in the branch destination path is “executed” as a return value (FIG. 12: step S83). Then, the process returns to the original process.

  On the other hand, when the read instruction is not “return” (step S65: No route), the determination unit 105 determines whether the read instruction is a branch instruction and is not included in the list of branch instructions that have been searched. Judgment is made (step S67). The reason for determining whether or not it is included in the list of branch instructions that have been searched is to prevent a loop. For example, data as shown in FIG. 13 is stored in the list of branch instructions that have been searched in the first instruction data storage unit 106.

  When the read instruction is a branch and is not included in the list of branch instructions that have been searched (step S67: Yes route), the determination unit 105 determines that the branch has been searched in the first instruction data storage unit 106. The read command is added to the command list (step S71).

  The determination unit 105 identifies one unprocessed branch destination (execution state is P) in the branch (step S73).

  The determination unit 105 executes determination processing (s, P, b) (step S75). Here, (s, P, b) is a list of instructions that have already been executed (that is, a list in the second instruction data storage unit 107) is in a state of s and has been searched (ie, a list of instructions that have been searched). In the case where the list in the first instruction data storage unit 106 is in a state of b, the execution state P is determined.

  As a result of the determination process, when it is determined that the instruction in the path in the execution state P has not been executed (step S77: No route), the process proceeds to step S81 in FIG.

  When it is determined that the instruction in the path in the execution state P has been executed (step S77: Yes route), the determination unit 105 determines whether there is an unprocessed branch destination (step S79). If there is an unprocessed branch destination (step S79: Yes route), the process returns to step S73 to process the next branch destination. On the other hand, when there is no unprocessed branch destination (step S79: No route), the process proceeds to step S69.

  On the other hand, if it is determined in step S67 that the read instruction is not a branch or is included in the list of branch instructions that have been searched (step S67: No route), the determination unit 105 determines the next of the execution state p. It is determined whether there is an instruction (step S69). If there is no instruction next to the execution state p (step S69: No route), the process proceeds to step S83 in FIG. On the other hand, if there is a next instruction in the execution state p (step S69: Yes route), the process returns to the process of step S61 in order to process the next instruction.

  If the above processing is executed, it can be determined whether or not there is an unexecuted instruction at the branch destination, so path pruning may be performed (that is, symbolic execution on the path may be terminated halfway). You can tell whether or not

  Returning to the description of FIG. 9, the execution unit 112 in the execution engine 104 determines whether the result of the determination process indicates that the instruction in the path of the execution state P is “executed” (step S47). When the instruction in the path of the execution state P has not been executed (step S47: No route), the process proceeds to step S51.

  On the other hand, when the path in the execution state P has been executed (step S47: Yes route), the execution unit 112 ends the symbolic execution for the path and adds the function being executed to the second function data storage unit 110. (Step S49). For example, as illustrated in FIG. 14, the second function data storage unit 110 stores a list of functions that have undergone path pruning.

  The execution unit 112 determines whether there is an unprocessed branch destination (step S53). When there is an unprocessed branch destination (step S53: Yes route), the processing returns to step S41 in order to process the next branch destination. On the other hand, when there is no unprocessed branch destination (step S53: No route), the process returns to the process of FIG.

  On the other hand, when it is determined in step S27 that the read instruction is not a branch related to the symbol variable (step S27: No route), the execution unit 112 executes the instruction and advances (ie, increments) the program counter. (Step S29).

  The execution unit 112 determines whether the execution state p is an end state (step S31). The end state is, for example, a state where the value of the program counter has reached the number of the last line in the program. If it is not in an end state (step S31: No route), the process returns to step S21 to process the next command.

  On the other hand, when the execution state p is an end state (step S31: Yes route), the generation unit 113 generates a specific value that satisfies the path condition obtained by the symbolic execution as test data (step S33), and the main memory Or the like in a storage device. Then, the process returns to the original process.

  If the above processing is executed, path pruning is executed when various conditions are satisfied, so symbolic execution is not performed for all paths as a target range, and the time required for symbolic execution can be reduced. The consumption of resources such as memory can be suppressed.

  Returning to the description of FIG. 3, the execution unit 112 in the execution engine 104 calculates coverage (step S9). For example, when an input of a target value for instruction coverage is accepted as the target coverage, the instruction coverage is calculated in step S9.

  The execution unit 112 determines whether the calculated coverage is larger than the target coverage (step S11). When it is below the target coverage (step S11: No route), the execution unit 112 requests the range control unit 103 to expand the target range of symbolic execution. In response to this, the range control unit 103 is a node that belongs to the (n−1) stage hierarchy from the leaf node in the call graph specified based on the data stored in the call graph data storage unit 111 and the previous time. The function that is the target of path pruning is added to the list of functions that are not the target of path pruning in the first function data storage unit 109 (step S13). Further, the range control unit 103 requests the execution engine 104 to perform symbolic execution again. Then, the process returns to step S3.

  For example, assume that a call graph as shown in FIG. 15 is obtained. When there is a function that calls a plurality of functions having different hierarchies as in func1 in FIG. 15, there are a plurality of methods for specifying a node belonging to the (n-1) stage hierarchy from the leaf node. . In such a case, the leaf node belonging to the lowest hierarchy is considered as a reference. In the example of FIG. 15, since func3 is considered as a reference instead of func4, func1 is a node belonging to a hierarchy two levels higher than the leaf node.

  If there is a recursive call in the program, there are no leaf nodes in the call graph, as shown on the left side of FIG. Therefore, when it is detected that a recursive call exists, an edge corresponding to the recursive call is deleted as shown on the right side of FIG. In order to detect the presence of a recursive call, for example, it is only necessary to check whether there is a function to be called in the call stack when the call graph is generated.

  On the other hand, when larger than the target coverage (step S11: Yes route), the generation unit 113 in the execution engine 104 stores the test data generated in step S33 in the test data storage unit 114 (step S15). The test data stored in the test data storage unit 114 is the test data that is finally adopted. Then, the process ends.

  By executing the processing as described above, it is possible to repeatedly perform symbolic execution while gradually reducing the range of path pruning and employ test data at the time when the coverage exceeds the target coverage. This makes it possible to efficiently generate test data with completeness.

  Here, a specific description will be given of a case where the processing of this embodiment is executed with the program shown in FIG. 4 as a target program. The target coverage is a target value of instruction coverage, and is assumed to be 90%.

  In the first symbolic execution, since the list of functions to be excluded from the path pruning is empty, all paths in the program become the path pruning target range. Since all the instructions of line numbers 15 to 19 are already executed in the middle of symbolic execution, path pruning is performed for “int string (char * s)”. As a result, the instructions of line numbers 11 to 14 are not executed. When the instruction coverage is calculated here, the instruction coverage becomes “(19-4) / 19 * 100≈79 (%), which is below the target coverage. Therefore, the target range of the path pruning is narrowed and symbolic execution is performed again. It will be.

  The functions that are not subject to path pruning are nodes that belong to the hierarchy (1-1 =) 0 level up from the leaf node (int strlen (char * s)) in the call graph (FIG. 8) and have been pruned the previous time. Since it is a function, it is “int string (char * s)”.

  In the second symbolic execution, since “int strlen (char * s)” is not subject to path pruning, instructions 11 to 14 are also executed. When the instruction coverage is calculated here, the instruction coverage is 19/19 * 100 = 100 (%), which is larger than the target coverage. Therefore, a specific value that satisfies the path condition obtained by the second symbolic execution is adopted as the test data.

  The same applies when an input of a target value of branch coverage or condition coverage is received as target coverage. For example, it is assumed that the target value of branch coverage is 90 (%). When the branch coverage is calculated after the first symbolic execution, 4/6 * 100≈67 (%), which is below the target coverage. Accordingly, as in the case of instruction coverage, symbolic execution is performed again with “int strlen (char * s)” as the function that is not subject to path pruning. Then, the branch coverage is 6/6 = 100 (%), which is larger than the target coverage. Therefore, a specific value that satisfies the path condition obtained by the second symbolic execution is adopted as the test data.

  The execution tree of the program shown in FIG. 4 is shown on the left side of FIG. As shown on the left of FIG. 17, there are six paths in the program shown in FIG. On the other hand, when the processing of the present embodiment is executed, as shown on the right side of FIG. When path pruning is performed in this way, the path coverage does not reach 100 (%), but as described above, the instruction coverage is 100 (%).

  In the above, the method for controlling the range of path pruning has been described. In the following, a case will be considered in which the range of path pruning is not controlled and all paths in the program are set as the range of path pruning.

  FIG. 18 shows a main processing flow in the case where the range of path cutting is not controlled. First, the input unit 100 in the symbolic execution device 1 receives an input of a target program to be subjected to symbolic execution from the user (FIG. 18: step S91). The input unit 100 stores the target program in the target program storage unit 102.

  The execution unit 112 in the execution engine 104 initializes the execution state p by initializing data related to the execution state stored in the state data storage unit 108 (step S93). The initialization means, for example, returning the value of the program counter to 1, clearing the contents of the call stack, clearing information about the state of the symbol variable from the memory, and clearing the pass condition.

  The execution unit 112 performs symbolic execution (null, p) (step S95). The symbolic execution (s, p) performed here will be described with reference to FIG. Note that the first character string in parentheses represents a list of executed instructions, and the second character string in parentheses is a character string representing an execution state. Here, (null, p) is to execute symbolic execution from the initial execution state p when the list of executed instructions (that is, the list in the second instruction data storage unit 107) is empty. Represents.

  First, the execution unit 112 reads one instruction from the target program stored in the target program storage unit 102 based on the program counter in the execution state p, and adds it to the second instruction data storage unit 107 (FIG. 19: S101). .

  The execution unit 112 determines whether the read instruction is a branch related to a symbol variable (step S103). The branch related to the symbol variable is, for example, “if (stren (s) <= 1)” in FIG.

  When the branch is not related to the symbol variable (step S103: No route), the execution unit 112 executes the instruction and advances (ie, increments) the program counter (step S117).

  The execution unit 112 determines whether the execution state p is an end state (step S119). The end state is, for example, a state where the value of the program counter has reached the number of the last line in the program. If it is not an end state (step S119: No route), the process returns to the process of step S101 in order to process the next instruction.

  On the other hand, when the execution state p is an end state (step S119: Yes route), the generation unit 113 generates, as test data, a specific value that satisfies the path condition obtained by symbolic execution (step S121). Or the like in a storage device. Then, the process returns to the original process.

  On the other hand, when it is determined in step S103 that the read instruction is a branch related to the symbol variable (step S103: Yes route), the execution unit 112 determines that the branch destination that has not been processed in the branch (the execution state is P). Is identified (step S105).

  The execution unit 112 requests the determination unit 105 to execute a determination process. In response, the determination unit 105 executes determination processing (s, P, null) (step S107). The determination process is as described with reference to FIGS. The first character string in the parentheses represents a list of instructions that have been executed, the second character string in the parentheses is a character string indicating an execution state, and the third character string in the parentheses is This represents a list of branch instructions that have been searched. Here, (s, P, null) is a list of instructions that have been executed (that is, a list in the second instruction data storage unit 107) that is already executed (ie, a list in the second instruction data storage unit 107) and that has already been searched (that is, In the state where the list in the first instruction data storage unit 106 is empty, the execution state P is determined.

  The execution unit 112 in the execution engine 104 determines whether the result of the determination process indicates that the instruction in the path of the execution state P is “executed” (step S109). When the instruction in the path in the execution state P has been executed (step S109: Yes route), the execution unit 112 ends the symbolic execution for the path (step S111).

  On the other hand, when the instruction in the path of the execution state P is not executed (step S109: No route), the execution unit 112 performs symbolic execution (s, P) (step S113). Here, (s, P) indicates that symbolic execution is performed for the execution state P when the list of executed instructions (that is, the list in the second instruction data storage unit 107) is in the state s. ing.

  The execution unit 112 determines whether there is an unprocessed branch destination (step S115). If there is an unprocessed branch destination (step S115: Yes route), the processing returns to step S105 to process the next branch destination. On the other hand, when there is no unprocessed branch destination (step S115: No route), the process returns to FIG. 18 and ends.

  By doing so, it is possible to set all the paths in the program as the target range of path pruning without controlling the range of path pruning.

  Even if such processing is performed, there is a case where omission can be executed for some paths while suppressing a decrease in instruction coverage. For example, in the case of the target program shown in FIG. 20, omission can be executed for some paths while suppressing a decrease in instruction coverage. The target program shown in FIG. 20 calls the function “int funcA (int a, int b)” in the code representing the main process, and the function “int abs (int a, int b)” in the function “int funcA (int a, int b)”. x) ". The number of instructions is 19. Note that the call stack during execution of any one of the instructions 1 to 7 is “stack3”, and the call stack during execution of any of the instructions 8 to 15 is “stack2”. The call stack during execution of any one of the instructions 16 to 19 is “stack1”.

  FIG. 21 shows an execution tree of the target program shown in FIG. In the execution tree in FIG. 21, the value of the program counter, the identification information of the call stack (corresponding to the identification information in FIG. 20), the information about the state of the symbol variable, and the path condition are shown for each execution state. Yes. When path pruning is performed on the target program shown in FIG. 20 by the processing described with reference to FIGS. 18 and 19, for paths corresponding to execution states 6 and 7, an instruction is issued by a path related to the condition a <0. Since it has been executed, the path pruning is performed. However, instruction coverage is not reduced.

  However, instruction coverage may be reduced depending on the structure of the target program. For example, in the case of the target program shown in FIG. 4, instruction coverage is reduced. The right side of FIG. 22 shows a result when path pruning is executed on the target program shown in FIG. 4 by the processing described in FIGS. 18 and 19. During symbolic execution, since instructions 15 to 19 are already executed in the path “! S [0]”, path pruning is performed on the part marked with a cross. Therefore, the path corresponding to the portion surrounded by the dotted line is not executed. The instruction coverage at this time is the same as the instruction coverage for the first symbolic execution when the range of path pruning is controlled, and is “(19-4) / 19 * 100≈79 (%)”.

  As described above, when the range of path pruning is not controlled and all paths in the program are set as the range of path pruning, instruction coverage (similar to condition coverage and branch coverage) may be reduced. Therefore, as described with reference to FIGS. 3 to 17, when the target range of path pruning is controlled, the path pruning can be performed so that the instruction coverage (same condition coverage and branch coverage) is equal to or greater than the target value. It is effective because it can.

  Although one embodiment of the present invention has been described above, the present invention is not limited to this. For example, the functional block configuration of the symbolic execution device 1 described above may not match the actual program module configuration.

  Further, the configuration of each table described above is an example, and the configuration as described above is not necessarily required. Further, in the processing flow, the processing order can be changed if the processing result does not change. Further, it may be executed in parallel.

  The symbolic execution device 1 described above is a computer device, and as shown in FIG. 23, a memory 2501, a CPU (Central Processing Unit) 2503, a hard disk drive (HDD: Hard Disk Drive) 2505, and a display device. A display control unit 2507 connected to 2509, a drive device 2513 for the removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for executing the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing content of the application program, and performs a predetermined operation. Further, data in the middle of processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In the embodiment of the present invention, an application program for performing the above-described processing is stored in a computer-readable removable disk 2511 and distributed, and installed in the HDD 2505 from the drive device 2513. In some cases, the HDD 2505 may be installed via a network such as the Internet and the communication control unit 2517. Such a computer apparatus realizes various functions as described above by organically cooperating hardware such as the CPU 2503 and the memory 2501 described above and programs such as the OS and application programs. .

  The embodiment of the present invention described above is summarized as follows.

  The test data generation method according to the present embodiment includes (A) a first process for performing symbolic execution while omitting a part of the target range of symbolic execution, and calculating a coverage rate for the symbolic execution; B) a second process for determining whether the calculated coverage rate exceeds a predetermined value; and (C) if the calculated coverage rate does not exceed the predetermined value, at least one of the omitted partial ranges. Obtained by symbolic execution when the calculated coverage rate exceeds a predetermined value, and a third process that executes the process after the first process again. And a fourth process of storing a value satisfying the condition of the path to be stored in the first data storage unit.

  In this way, since the coverage rate exceeds a predetermined value, it is only necessary to perform symbolic execution at a minimum, so that it is possible to efficiently generate test data with completeness.

  In the first process described above, (a1) when there is no unexecuted instruction in each of a plurality of branch destinations generated by the branch in the program, the symbolic execution is terminated for the path related to the branch, Some ranges may be omitted and symbolic execution may be performed. In this way, paths that are unlikely to contribute to the improvement of the coverage rate can be omitted, so that the time required for symbolic execution can be shortened and the consumption of computer resources can be suppressed.

  In the third process described above, (c1) functions specified by a predetermined method among functions included in a part of the omitted range may be determined to be excluded. As described above, if the exclusion target is determined in units of functions, the range to be excluded from the suppression target can be appropriately determined.

  Further, the predetermined method described above may be a method of specifying sequentially from the function belonging to the lowest layer in the call graph generated based on the function call relationship. In this way, the omitted range can be gradually reduced.

  The coverage rate described above may include at least one of an instruction coverage rate, a condition coverage rate, and a branch coverage rate. Normal symbolic execution is performed so that the route coverage rate is 100%. However, if the instruction coverage rate, condition coverage rate, or branch coverage rate only needs to be a predetermined value, normal symbolic execution is performed. There is a lot of waste. Therefore, if the coverage rate described above is adopted as the coverage rate, it is not necessary to perform symbolic execution wastefully.

  A program for causing a computer to perform the processing according to the above method can be created. The program can be a computer-readable storage medium such as a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, a hard disk, or the like. It is stored in a storage device. The intermediate processing result is temporarily stored in a storage device such as a main memory.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1)
A first process of performing a symbolic execution while omitting a part of a target range of the symbolic execution, and calculating a coverage rate for the symbolic execution;
A second process for determining whether the calculated coverage ratio exceeds a predetermined value;
If the calculated coverage ratio does not exceed the predetermined value, set so that at least a part of the omitted partial range is not subject to omission, and the processes after the first process are performed. A third process to be executed again;
A fourth process of storing, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generation method in which a computer executes.

(Appendix 2)
In the first process,
When there is no unexecuted instruction in each of a plurality of branch destinations generated by a branch in the program, the symbolic execution is omitted by ending the symbolic execution for the path related to the branch, thereby omitting the partial range. The test data generation method according to appendix 1, wherein:

(Appendix 3)
In the third process,
The test data generation method according to appendix 1 or 2, wherein a function specified by a predetermined method among the functions included in the omitted partial range is determined to be excluded.

(Appendix 4)
The predetermined method is:
4. The test data generation method according to appendix 3, wherein the test graph is specified in order from a function belonging to the lowest layer in a call graph generated based on a function call relationship.

(Appendix 5)
The coverage rate is
The test data generation method according to any one of appendices 1 to 4, wherein the test data generation method includes at least one of an instruction coverage rate, a condition coverage rate, and a branch coverage rate.

(Appendix 6)
A first process of performing a symbolic execution while omitting a part of a target range of the symbolic execution, and calculating a coverage rate for the symbolic execution;
A second process for determining whether the calculated coverage ratio exceeds a predetermined value;
If the calculated coverage ratio does not exceed the predetermined value, set so that at least a part of the omitted partial range is not subject to omission, and the processes after the first process are performed. A third process to be executed again;
A fourth process of storing, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generation program for causing a computer to execute.

(Appendix 7)
First processing for performing symbolic execution while omitting a part of the target range of symbolic execution, calculating a coverage rate for the symbolic execution, and determining whether the calculated coverage rate exceeds a predetermined value And
If the calculated coverage ratio does not exceed the predetermined value, at least a part of the omitted part of the range is set so as not to be omitted, and the first processing unit performs processing. A second processing unit to be executed again;
A third processing unit that stores, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generating apparatus having

DESCRIPTION OF SYMBOLS 1 Symbolic execution apparatus 100 Input part 101 Target coverage storage part 102 Target program storage part 103 Range control part 104 Execution engine 105 Judgment part 106 1st command data storage part 107 2nd command data storage part 108 State data storage part 109 1st function Data storage unit 110 Second function data storage unit 111 Call graph data storage unit 112 Execution unit 113 Generation unit 114 Test data storage unit

Claims (6)

A first process of performing a symbolic execution while omitting a part of a target range of the symbolic execution, and calculating a coverage rate for the symbolic execution;
A second process for determining whether the calculated coverage ratio exceeds a predetermined value;
If the calculated coverage ratio does not exceed the predetermined value, set so that at least a part of the omitted partial range is not subject to omission, and the processes after the first process are performed. A third process to be executed again;
A fourth process of storing, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generation method in which a computer executes. In the first process,
When there is no unexecuted instruction in each of a plurality of branch destinations generated by a branch in the program, the symbolic execution is omitted by ending the symbolic execution for the path related to the branch, thereby omitting the partial range. The test data generation method according to claim 1, wherein: In the third process,
3. The test data generation method according to claim 1, wherein a function specified by a predetermined method among the functions included in the omitted partial range is determined to be excluded. The predetermined method is:
4. The test data generation method according to claim 3, wherein the test data is specified in order from a function belonging to the lowest layer in a call graph generated based on a function call relationship. A first process of performing a symbolic execution while omitting a part of a target range of the symbolic execution, and calculating a coverage rate for the symbolic execution;
A second process for determining whether the calculated coverage ratio exceeds a predetermined value;
If the calculated coverage ratio does not exceed the predetermined value, set so that at least a part of the omitted partial range is not subject to omission, and the processes after the first process are performed. A third process to be executed again;
A fourth process of storing, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generation program for causing a computer to execute. First processing for performing symbolic execution while omitting a part of the target range of symbolic execution, calculating a coverage rate for the symbolic execution, and determining whether the calculated coverage rate exceeds a predetermined value And
If the calculated coverage ratio does not exceed the predetermined value, at least a part of the omitted part of the range is set so as not to be omitted, and the first processing unit performs processing. A second processing unit to be executed again;
A third processing unit that stores, in the first data storage unit, a value that satisfies a path condition obtained by the symbolic execution when the calculated coverage rate exceeds the predetermined value;
A test data generating apparatus having

Images