JP2010267022A - Test data generation method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate test data allowing intensive detection of trouble having high occurrence frequency, and allowing unique specification of a variable causing the trouble. <P>SOLUTION: When generating the test data including a pair of an input (a combination between a factor and a level) and an expectation value, a combination between the inputs not but combinations between all the levels possible to the factor is generated such that each of all the levels in each factor is used in a test at least once. As the combination between the inputs, a combination (a normal input) including all normal levels and a combination (an abnormal input) between the factor and the level including only one of abnormal levels are generated. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a test data generation method, apparatus, and program, and in particular, test data used in a test for confirming whether or not a system has been constructed in accordance with a design intent described in a design model. The present invention relates to a test data generation method, apparatus, and program for automatically generating a design model rather than using a design model. Specifically, the present invention relates to a test data generation method, apparatus, and program for identifying the cause of a defect.

  In the test in the program development process, whether or not the system is constructed as designed is confirmed, and test data is required at that time. If test data can be prepared automatically instead of manpower, development costs can be improved. In addition, if the test can be carried out effectively with a small amount of test data, the test cost can be further reduced.

  Test data that can be effectively tested is considered to be test data that makes it easier to detect a defect, and test data that makes it easier to identify the cause when a defect is detected.

  Test data consists of combinations of values taken by a plurality of variables. If all the possible combinations are generated as test data, there is a problem that when the number of variables increases, the combination explosion makes processing impossible. For this reason, usually, only a part of all combinations is used as test data. However, a method for narrowing down the number of combinations is required at this time.

  As test data automatic generation technology, there are, for example, a method of decomposing a data type having a hierarchical structure into a basic type and giving a predetermined value to the basic type, and a technology of generating a value to be assigned to a single variable (for example, Patent Document 1, Non-Patent Document 1).

JP-A-9-231103

Jon Edvardsson. "A Survey on Automatic Test Data Generation". In Proceedings of the 2nd Conference on Computer Science and Engineering, Pages 21-28, 1999. http://citeseerx.ist.psu.edu/viewdoc/summary?doi= 10.1.1.20.963

  However, many conventional test data automatic generation techniques do not mention the problem of a combination of values taken by the plurality of variables, or generate a combination randomly without a policy. In the techniques of Patent Document 1 and Non-Patent Document 1 described above, generation of a value to be assigned to a single variable is described. However, when there are a plurality of variables, a specific combination is generated. Not done.

  As described above, there is a problem that even if the conventional automatic test data generation technology is combined, it is easy to detect a defect even with a small amount of test data, and it is not possible to easily identify the cause when a defect is detected. .

  The present invention has been made in view of the above points, and in the automatic generation of test data to a system based on a design model including structure and object type data, it is possible to focus on frequently occurring defects. It is another object of the present invention to provide a test data generation method, apparatus, and program that can uniquely specify a variable that causes a failure.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1) is a test data generation device for generating test data for confirming whether or not the design information of a system is constructed in accordance with the design intention of a design model described in a formal language. And
A design model reading means 1 for reading a design model described only by a UML class diagram and an activity diagram from the user terminal 20;
Design model analysis means 2 for extracting an activity diagram as a test object from the read design model;
Execution path extraction means 3 for extracting an execution path from the test target;
Test data specification storage means 12 storing test data specifications;
Constraints extracted by scanning the end-point node from the start-point node of the execution path that the variable (“factor” in the experimental design method) should satisfy the input parameters of the activity having each execution path extracted from the execution-path extraction means 4 The test data specification generation means 4 for updating the test data specification stored in the test data specification storage means 12,
The test data specification is read from the test data specification storage means 12, and a normal level allowed for the execution path according to the value of the test data specification as a value taken by the factor ("level" in the experimental design method) and execution Level generation means 5 for generating an abnormal level that is not allowed for the path, and storing the factor and the level in the execution path / level storage means 11 in correspondence with the execution path;
It is a combination of a normal input that is a combination of only the factor and the normal level, and a level that includes only one of the factor and the abnormal level, by acquiring the factor, the level, and the execution path from the execution path / level storage unit 11. Test data that generates an abnormal input, obtains an expected value based on whether or not a postcondition is included in the execution path, generates test data with the expected value added to the input, and stores the test data in the test data storage unit 13 Generating means 6;
Output means 7 for outputting test data to the user terminal 20.

  FIG. 2 is a diagram for explaining the principle of the present invention.

The present invention (Claim 2) is a test data generation method for generating test data for confirming whether or not the system design information is constructed in accordance with the design intent of the design model described in a formal language. And
A design model reading step (step 1) for reading a design model described only by a UML class diagram and an activity diagram from a user terminal;
A design model analysis step (step 2) for extracting an activity diagram as a test target from the read design model;
An execution path extraction step (step 3) for extracting an execution path from the test target;
The input parameters of the activity with each execution path extracted in the execution path extraction step are the constraints extracted by scanning the end node from the start node of the execution path that the variable ("factor" in the experimental design method) should satisfy A test data specification generation step (step 4) for updating the test data specification stored in the test data specification storage means,
The test data specification is read from the test data specification storage means, and the normal level allowed for the execution path according to the value of the test data specification as the value of the factor ("level" in the experimental design method) and the execution A level generation step (step 5) for generating an abnormal level that is not allowed for the path, and storing the factor and the level in the execution path / level storage means in correspondence with the execution path;
An abnormality that is a combination of a normal input that is a combination of only the factor and the normal level, and a level that includes only one of the factor and the abnormal level, by acquiring the factor, the level, and the execution path from the execution path / level storage means. An input is generated (step 6), an expected value is obtained based on whether or not a postcondition is included in the execution path, test data with the expected value added to the input is generated (step 7), and test data is stored A test data generation step to be stored in the means;
An output step (step 8) of outputting test data to the user terminal is performed.

  The present invention (Claim 3) is a test data generation program for causing a computer to function as each means constituting the test data generation apparatus according to any one of Claims 1.

  As described above, according to the present invention, in the test, rather than a defect caused by a combination of values (“value” in the term of the experimental design method) of a plurality of variables (“factor” in the term of the experimental design method) An empirical fact that there are many failures due to the value of one variable (eg, literature: RD Kuhn et al. "Software Fault Interactions and Implications for Software Testing." IEEE Transactions on Softweare Engineering, 30 (6), June 2004. ) To generate a “normal input” that is a combination of only a factor and a normal level, and an “abnormal input” that is a combination of a level that includes only one of the factor and an abnormal level. By narrowing down the number of data, it is possible to detect defects that occur frequently with a smaller number of test data. In other words, it is possible to focus on frequently occurring defects by using automatically generated test data.

  Furthermore, according to the present invention, it is possible to uniquely identify a variable that causes a defect by generating highly useful test data that facilitates identifying the cause of the defect. That is, when a failure is detected by certain test data input to the system, it is possible to identify which variable has the cause among the variables constituting the test data.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the test data generation device in one embodiment of the present invention. It is a figure which shows the UML design model image used by this invention. It is a figure which shows the format of the test data specification in one embodiment of this invention. It is an input-output image of the test data generation device in one embodiment of the present invention. It is a flowchart of the test data generation process outline | summary in one embodiment of this invention. It is an example of the execution path | route extracted by the execution path | route extraction part in one embodiment of this invention. It is a flowchart of the test data specification production | generation part in one embodiment of this invention. It is a figure for demonstrating the execution path | route in one embodiment of this invention. It is a figure which shows the production | generation method of the normal input and abnormal input in one embodiment of this invention. It is a flowchart of the test data generation part in one embodiment of this invention. It is a flowchart of the "generation of normal input" process of the test data generation unit in one embodiment of the present invention. It is a program level flowchart of a "normal input generation" process of a test data generation unit in an embodiment of the present invention. It is a flowchart of the "abnormal input generation" process of the test data generation part in one embodiment of this invention. It is a program level flowchart of the "abnormal input generation" process of the test data generation part in one embodiment of this invention. It is a flowchart of the test data generation section "giving expected value to input" processing in an embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, terms used in the following embodiments will be described.

  A “design model” is a description of system design information strictly and formally in some formal description language rather than a vague natural language.

  The “constraint condition” refers to a condition relating to a value that can be taken by a certain factor (here, a), such as “a> 3”, included in the execution path extracted from the design model. In the example of “a> 3”, the constraint condition is composed of a factor name (a), an operator (>), and an operand (3), and “invariant condition” and “precondition” depending on the location where the constraint condition is attached. ”And“ branching conditions ”. For the operator of the constraint condition, a commonly used notation such as an inequality sign or a character string comparison is used.

  The “test data specification” is a collection of properties to be satisfied by factors (variables) for passing through a specific execution path in the design model, and indicates “specification” that the test data should satisfy.

  “Updating the test data specification” means overwriting the test data specification from the initial value immediately after generation according to the processing of the constraint condition in the execution path.

  The “normal level” is an input value allowed for the execution path.

  The “abnormal level” is an input value that is not allowed for the execution path.

  “Normal input” is an input composed only of a factor and a normal level.

  The “abnormal level” is an input including a factor and one abnormal level.

  FIG. 3 shows a configuration of the test data generation apparatus according to the embodiment of the present invention.

  The test data generation device 10 shown in FIG. 1 includes a design model reading unit 1, a design model analysis unit 2, an execution path extraction unit 3, a test data specification generation unit 4, a level generation unit 5, a test data generation unit 6, and an output unit 7. , An execution path / level storage unit 11, a test data specification storage unit 12, and a test data storage unit 13.

  Among these, the design model reading unit 1, the design model analysis unit 2, the execution path extraction unit 3, the test data specification generation unit 4, the level generation unit 5, the test data generation unit 6, and the output unit 7 are functions on the CPU. The execution path / level storage unit 11, the test data specification storage unit 12, and the test data storage unit 13 are storage media such as a hard disk device.

  The user terminal 20 includes a modeling tool 21 and a viewer 22, and exports a design model from the modeling tool 21 to the design model reading unit 1 of the test data generation device. The viewer 22 acquires test data from the test data generation device 10.

  The design model reading unit 1 reads the XML model UML model (class diagram, activity diagram) as shown in FIG. 4 exported from the modeling tool 21 of the user terminal 20.

  The design model analysis unit 2 analyzes the read UML model and extracts activities as test targets.

  The execution path extraction unit 3 extracts an execution path (Path) from the test target, and outputs the extracted execution path to the test data generation unit 4 and the execution path / level storage unit 11.

  The test data specification generation unit 4 collects the properties to be satisfied by variables (“factors” in the experimental design method) for passing a specific execution path from all the execution paths extracted by the execution path extraction unit 3, It is stored in the data specification storage unit 12. As shown in FIG. 5, the test data specification differs for each data type (Integer type, Enum type, Object type, Boolean type, String type Collection type) and has a domain that defines factors. As the domain is updated (overwritten), the range is narrowed. For example, the test data specification for a String type factor includes values such as a lower limit, an upper limit, and a substring. The factor "name: String" has a length of 0 to 30, Test data specifications limit the possible values of a factor (here "name"), such as including the column "M".

  The level generation unit 5 reads the test data specification from the test data specification storage unit 12, generates a value of the factor (“level” in the experimental design method), and associates it with the factor and the execution path (Path Name). Stored in the execution path / level storage unit 11. The level generation unit 5 generates a normal level or an abnormal level based on the value indicated by the test data specification.

  The test data generation unit 6 acquires the execution path and level from the execution path / level storage unit 11 and generates test data including a pair of an input (a set of factors and levels) and an expected value, as shown in FIG. The data is stored in the test data storage unit 13 in the format shown. As shown in FIG. 6B, the test data includes items of an execution path name, test data ID, level (value), factor (variable), expected value, normal input or abnormal input distinction, and test result.

  The output unit 7 reads the test data generated by the test data generation unit 6 from the test data storage unit 13 and outputs it to the user terminal 20. Here, the output is in CSV format.

  The operation in the above configuration will be described below.

  FIG. 7 is a flowchart of the operation of the test data generation apparatus in one embodiment of the present invention.

  Step 100) The design model reading unit 1 reads an XML (Unified Modeling Language) model in XML format exported from the user terminal 200 and passes it to the design model analysis unit 2.

  Step 110) The design model analysis unit 2 decomposes the UML design model acquired from the design model reading unit 1, extracts activities as test targets, and passes them to the execution path extraction unit 3.

  Step 120) As shown in FIG. 8, the execution path extraction unit 3 scans from the start node to the end node of each acquired test target (activity) to extract the execution path, and extracts all execution path information from the execution path / The data is passed to the level storage unit 11 and the test data specification generation unit 4. The execution path information includes processing of each node, branch conditions, and post-conditions as shown in FIG.

  Step 130) The test data specification generation unit 4 generates a test data specification in a format as shown in FIG. 5 based on each execution path information, and stores it in the test data specification storage unit 12.

  Below, operation | movement of the test data specification production | generation part 4 is demonstrated. FIG. 9 is a flowchart of the test data specification generation unit in one embodiment of the present invention. In the following, it is assumed that the test data specification storage unit 12 is initialized by default with the corresponding test data specification based on the type of factor.

    Step 131) The test data specification generation unit 4 acquires execution path information from the execution path extraction unit 3 and stores it in a memory (not shown).

    Step 132) An execution path is acquired from a memory (not shown), and the execution path as shown in FIG. 10 is scanned from the start node to the end node, and the constraint conditions contained therein are extracted.

    Step 133 One test data specification is read from the test data specification storage unit 12. When all the test data specifications have been read, the process ends.

    Step 134) The factor name of the read test data specification is compared with the factor included in the constraint condition. If they match, the process proceeds to Step 135, and if they do not match, the process returns to Step 133.

    Step 135) If the factor names match, the test data specification in the test data specification storage unit 12 is updated with the contents of the constraint condition.

  Step 140) The level generation unit 5 reads the test data specification from the test data specification storage unit 12, generates a value taken by the factor ("level" in the experimental design method), and stores it in the execution path / level storage unit 11 . The level depends on the type of test data specification.

  For example, when the type of the test data specification is “Boolean type”, the value v of the Boolean type test data specification shown in FIG. 5 is acquired, and “true” and “false” are set to the normal level. Generate a non-existing value as an abnormal level

  In the case of the Enum type test data specification, each element s of the “allowable element S” of the Enum type test data specification shown in FIG. 5 is set to a normal level, and the element s ′ not belonging to S, null, and Enum type are not used. The value is an abnormal level.

  In the case of the Integer type test data specification, the lower limit L and the upper limit U of the Integer type test data specification shown in FIG. 5 are acquired, and L, L-1, U-1, U, and a random value X (L ≦ X ≦ U) is a normal level, L-1, U + 1, positive and negative values that overflow, null, and non-integer values are abnormal levels.

  For other types, a predetermined level is obtained for each type of test data specification.

  The level obtained as described above is stored in the execution path / level storage unit 11 together with the factor.

  Step 150) The test data generation unit 6 obtains a set of levels and factors (input) from the execution path / level storage unit 11, and generates test data consisting of pairs of inputs and expected values, as shown in FIG. Is stored in the test data storage unit 13 in the format shown in FIG. The “input” is a combination of a factor and a level acquired from the execution path / level storage unit 11. As an “input”, a combination of all normal levels for one factor is “normal input”, and a combination of a normal level and one abnormal level is “abnormal input” for one factor. As shown in FIG. 11A, a combination of normal levels is set as a normal input, and a combination including a normal level and one abnormal level is set as an abnormal level as shown in FIG.

  A method for generating the input will be described in detail with reference to FIG.

  Step 160) The output unit 7 reads the test data (CSV file) generated by the test data generation unit 6 from the test data storage unit 13 and outputs it to an external file (user terminal viewer 220).

  Next, the process of the test data generation unit 6 in step 150 will be described.

  FIG. 12 is a flowchart of the test data generation unit in one embodiment of the present invention.

  Step 151) All levels of all factors are acquired from the execution path / level storage unit 11 and stored in a memory (not shown).

  Step 152) A normal input consisting of a normal level is generated for each factor. The detailed generation process of the normal input is shown in FIG. 13, and an example of normal input is shown in FIG.

    Step 1521) The normal level number M of the factor A having the most normal level is specified from all the factors acquired in Step 151.

    Step 1522) M empty normal inputs are generated on the test data storage unit 13 (a record is generated).

    Step 1523) The normal level of factor A is sequentially filled in the normal input column of the generated record.

    Step 1524) Next, the normal level of each factor other than factor A is repeatedly filled in the normal input.

  Here, the input combinations are made so that all levels in each factor are used at least once in the test, rather than all possible combinations. This criterion is called “all-level coverage”. A combination consisting of all normal levels is defined as “normal input”. Therefore, the number of normal inputs generated coincides with the number of levels of the factor with the highest number of levels, which is the minimum number of combinations that satisfies all level coverage.

  When the above processing is expressed at the program level, the flow is as shown in FIG.

  Step 153) For each factor, an abnormal input including one abnormal level is generated. The abnormal input generation process will be described below with reference to FIG. An example thereof is shown in FIG.

    Step 1531) The total number N of abnormal levels of each factor is specified from all factors stored in a memory (not shown).

    Step 1532) N empty input entries are generated on the test data storage unit 13 (a record is generated).

    Step 1533) The generated normal input is repeatedly overwritten on the abnormal input on the record.

    Step 1534) Next, for each abnormal level, the abnormal level is overwritten on only one corresponding portion of the abnormal input based on which factor.

  In this process, a combination is generated so that only one abnormal level is included in one combination, not the minimum number of combinations that satisfy all levels. This is in order to be able to specify which factor causes the malfunction when a malfunction of the system is discovered due to an abnormal input.

  When the above processing is expressed at the program level, the flow is as shown in FIG.

  Step 154) An expected value is assigned to each input.

  Below, giving of an expected value is demonstrated along FIG.

  FIG. 17 is a flowchart of the “applying expected value to input” process of the test data generation unit according to the embodiment of the present invention.

    Step 1541) An input (factor + level) and an execution path corresponding to the input are acquired from the execution path / level storage unit 12.

    Step 1542) All postconditions in the execution path are acquired. As shown in FIG. 10, the post-condition can be acquired by checking the nodes in order from the start node to the end node of the execution path.

    Step 1543) If the input (factor + level) is a normal input, the process proceeds to step 1544; otherwise, the process proceeds to step 1545.

    Step 1544) If the post-conditions can be acquired in step 1542, the process proceeds to step 1547. If the post-conditions cannot be acquired, the process proceeds to step 1546.

    Step 1545) The expected value is set to “undefined” and given to the input (factor + level).

    Step 1546) The expected value is set as “undefined” and given to the input (factor + level).

    Step 1457) The expected value is assigned to the input (factor + level) as “a set of post-conditions”.

In the example shown in FIG. 6, [status = APPROVED, orderld = arbitrary character string (999 characters)]
Is one input and [changedOrder has been added] is the expected value.

  The operation of each component of the test data generation device shown in FIG. 3 can be constructed as a program and can be installed and executed on a computer used as the test data generation device, or distributed via a network. Is possible.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiments and examples, and various modifications and applications can be made within the scope of the claims.

1 design model reading means, design model reading section 2 design model analyzing means, design model analyzing section 3 execution path extracting means, execution path extracting section 4 test data specification generating means, test data specification generating section 5 level generating means, level generating section 6 Test Data Generation Unit, Test Data Generation Unit 7 Output Unit, Output Unit 10 Test Data Generation Device 11 Execution Path / Level Storage Unit, Execution Path / Level Storage Unit 12 Test Data Specification Storage Unit, Test Data Specification Storage Unit 13 Test Data Storage means, test data storage unit 20 User terminal 21 Modeling tool 22 Viewer

Claims (3)

A test data generation device for generating test data for confirming whether or not the system design information is constructed according to the design intention of a design model described in a formal language,
A design model reading means for reading a design model described only by a UML class diagram and an activity diagram from a user terminal;
Design model analysis means for extracting an activity diagram as a test target from the read design model;
Execution path extraction means for extracting an execution path from the test target;
Test data specification storage means storing test data specifications;
Constraint conditions extracted by scanning the end point node from the start point node of the execution path that the variable ("factor" in the experimental design method) should satisfy the input parameters of the activity having each execution path extracted from the execution path extracting means Based on the test data specification storage means for updating the test data specification stored in the test data specification storage means,
The test data specification is read from the test data specification storage means, and according to the value of the test data specification, as a value taken by the factor ("level" in the experimental design method), a normal level allowed for the execution path, Level generation means for generating an abnormal level that is not allowed for the execution path, and storing the factor and the level in the execution path / level storage means in correspondence with the execution path;
A combination of a level including only one of the factor and the abnormal level obtained by acquiring the factor, the level and the execution path from the execution path / level storage means, and a normal input which is a combination of the factor and the normal level only. A test for generating an abnormal input, obtaining an expected value based on whether or not a postcondition is included in the execution path, generating test data with the expected value added to the input, and storing the test data in the test data storage means Data generation means;
Output means for outputting the test data to the user terminal;
A test data generation apparatus comprising: A test data generation method for generating test data for confirming whether or not the design information of a system is constructed according to the design intention of a design model described in a formal language,
A design model reading step for reading a design model described only by a UML class diagram and an activity diagram from a user terminal;
A design model analysis step of extracting an activity diagram from the read design model as a test target;
An execution path extraction step for extracting an execution path from the test target;
Constraint conditions extracted by scanning the end node from the start node of the execution path that the variable ("factor" in the experimental design method) should satisfy the input parameters of the activity having each execution path extracted in the execution path extraction step And a test data specification generation step for updating the test data specification stored in the test data specification storage means,
The test data specification is read from the test data specification storage means, and according to the value of the test data specification, as a value taken by the factor (“level” in the experimental design method), a normal level allowed for the execution path, A level generation step of generating an abnormal level that is not allowed for the execution path, and storing the factor and the level in the execution path / level storage unit in association with the execution path;
A combination of a level including only one of the factor and the abnormal level obtained by acquiring the factor, the level and the execution path from the execution path / level storage means, and a normal input which is a combination of the factor and the normal level only. A test that generates an abnormal input that is, calculates an expected value based on whether or not a postcondition is included in the execution path, generates test data with the expected value added to the input, and stores the test data in a test data storage unit A data generation step;
Outputting the test data to the user terminal;
The test data generation method characterized by performing.   A test data generation program for causing a computer to function as each means constituting the test data generation apparatus according to claim 1.

Images