JP6369102B2 - Software test support system - Google Patents

Description

  The present invention relates to a system for supporting software testing work.

  In general, in order to perform a program operation test, input information (arguments, etc .; test pattern) required when the program operates is input, processing results for this input information are received as output information, and this output information To identify whether the input information has been correctly processed by the program. For this reason, automatic evaluation methods that enable efficient confirmation of program defects have been studied.

  On the other hand, a general test method (black box test or white box test) is used when performing a software test, but as the scale of the program grows, trying to check the operation for all input information, Since the number of confirmations is enormous, there are test techniques for reducing the number of confirmations. For example, an all-pair method (pair-wise method), a method using an orthogonal table (HAYST method), an equivalence division method, a boundary value analysis method, and the like. By using these test techniques, it is possible to reduce the number of program operation checks.

For example, conventional techniques disclosed in Patent Documents 1 and 2 are known.
The invention of Patent Document 1 can generate a reasonable amount of test cases that can be used to check the quality of a software program in consideration of business specifications. Patent Document 1 discloses generating a test case based on an all-pair method, a method using an orthogonal table, or the like. Furthermore, there is a disclosure that a test case generation tool such as PICT (Pairwise Independent Combinatorial tool) can be used when generating a test case.

  Moreover, in the invention of Patent Document 2, an operation test of an application that operates by being incorporated in an ERP package can be efficiently executed.

JP 2012-248101 A JP 2011-164788 A

  By applying an appropriate test technique to the test target program, it is possible to reduce the number of times of operation confirmation of the test target program. However, in order to determine an appropriate test technique, it is necessary to have a detailed understanding of the various test techniques, and there are many verification procedures that are always applied to actual software tests. The person who is familiar with is also necessary. In addition, human error is likely to occur due to the high level of difficulty of applying appropriate test techniques.

  An object of the present invention is to automatically determine an appropriate test technique according to a test target program, and further to automatically generate a test pattern according to the test technique, thereby reducing the number of operation confirmations. It is to provide a software test support system etc.

The software test support system of the present invention obtains the input condition generation means for generating an input condition based on the specification information of the target program and the number of combinations corresponding to the input condition for each of the various registered test techniques. A test technique determining means for determining a test technique with the smallest number of combinations as a test technique to be applied to the target program; and a test pattern generating means for generating a test pattern for the target program according to the determined test technique. .

  According to the software test support system and the like of the present invention, it is possible to automatically determine an appropriate test technique corresponding to the test target program, and further to automatically generate a test pattern corresponding to the test technique. The number of operation confirmations can be reduced.

It is a figure which shows the function structural example of the test assistance system of this example. It is a process flowchart figure of the test assistance system of FIG. It is a figure which shows an example of program specification information. It is a figure which shows the bit numerical value of Q12 format expressing "0.5". (A) is program specification information, (b) is a specific example of the calculation result of the number of combinations. It is a specific example of the generated combination conditions. It is a specific example of the input information (test pattern) produced | generated.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram illustrating a functional configuration example of the test support system 10 of the present example.
The test support system 10 in the illustrated example includes various processing function units such as an input unit 11, an input condition determination unit 12, a test technique determination unit 13, a test technique application unit 14, an input information generation unit 15, and an output unit 16.

  The test support system 10 is realized on a general general-purpose computer such as a personal computer or a server device. Therefore, although not particularly illustrated, the test support system 10 includes a CPU (not shown), a storage unit (memory, hard disk, etc.), an input / output interface, a communication function unit, and the like in hardware. A predetermined application program is stored in the storage unit in advance. When the CPU executes the application program, the various processing function units shown in FIG. 1 and the processing of the flowchart shown in FIG.

  In general, the test support system 10 inputs the specification information 1 of the processing target program via the input unit 11 and determines a test technique to be applied to the test of the processing target program based on the specification information 1. Then, input information (test pattern) corresponding to the test technique is generated and output from the output unit 16 as the input information file 2. Then, the test target program is tested using this test pattern in a test system (not shown) or the like. The number of test patterns corresponds to the number of operation confirmations.

This will be described in more detail with reference to the flowchart of FIG.
When the input condition determination unit 12 acquires the program specification information 1 via the input unit 11 (step S1), the input condition determination unit 12 determines an input condition based on the program specification information 1 (step S2).

  The specification information 1 will be described later using a specific example shown in FIG. 3. For example, each function of the processing target program includes information such as a variable type, a data format, and a branch condition for each variable. It is. The input condition will be described later using a specific example shown in FIG. 5A. For example, each input variable is a data group determined based on the variable type, branch condition, and the like.

  The test technique determination unit 13 determines a test technique to be applied to the processing target program based on the input conditions and the like (step S3). This is to select a test technique that minimizes the number of operation confirmations (number of combinations) according to the input conditions from among a plurality of types of test techniques registered in advance. In this example, there are three types of test techniques registered in advance: the round robin method, the all-pair method, and the method using an orthogonal table (HAYST method). Absent.

The application test technique determination process performed by the test technique determination unit 13 will be described later with reference to specific examples shown in FIGS.
The test technique application unit 14 generates a combination condition related to the applied test technique determined by the test technique determination unit 13 (step S4). You may consider that this combination condition is a test pattern substantially. However, there is a case where it is necessary to convert the data into a predetermined data format as necessary.

  The test technique application unit 14 has an existing combination condition generation function for each of the various test techniques (brute force method, all-pair method, HAYST method), and generates a combination condition related to the processing target program by an arbitrary test technique. . This can be generated by the functionality of existing tools. For example, with respect to the all-pair method, a combination condition related to the processing target program can be generated by the function of an existing tool such as PICT described above. A specific example of the combination condition is shown in FIG. 6 and will be described later.

  The combination condition generation function may also be possessed by the test technique determination unit 13. This is because if the combination condition is generated, the number of combinations is also known. However, in this description, as described later, an example in which the number of combinations can be obtained without generating a combination condition will be described.

  The input information generation unit 15 generates input information (test pattern) used for the test based on the combination conditions related to the applied test technique (step S5). This is for generating the input information shown in FIG. 7 based on the combination condition of FIG. Note that the input information generation unit 15 performs processing for conversion to a predetermined data format as necessary, and as described above, the combination condition may be regarded as a substantial test pattern. .

The generated input information (test pattern) is output from the output unit 16 as the input information file 2 (step S6).
In some cases, the combination condition may be used as the input information (test pattern) as it is. This is the case, for example, when the data format of output related information described later is not particularly specified. In the example of FIG. 3 described later, since Q12 is designated as the data format, the input information (test pattern) is generated by converting the numerical value in the combination condition into the Q12 format. Details will be described later.

  From the above, the test technique application unit 14 may be regarded as an input information (test pattern) generation function unit, and the input information generation unit 15 may be regarded as a data format conversion function unit. . Further, the input information generation unit 15 may be regarded as being included in the test technique application unit 14. From the above, the test technique application unit 14 may be regarded as having an existing test pattern generation function for each of the various test techniques (brute force method, all-pair method, HAYST method).

  As described above, an appropriate test technique is automatically determined, and a test pattern corresponding to the determined test technique is generated and output. As described above, the number of test patterns is the smallest, and the number of operation confirmations is minimized. Although not shown, naturally, some test system uses the input information file 2 to perform an operation test of the software to be tested.

Hereinafter, a more detailed description will be given with reference to specific examples in FIG.
FIG. 3 shows an example of the program specification information 1. The program specification information 1 is arbitrarily created and registered in advance by, for example, a program developer, but is not limited to this example.

The program specification information 1 includes a function name, input related information, output related information, and the like, and the name, input related information, output related information, and the like are registered for each function of the processing target program.
The input related information includes a variable name, a variable type, a data format, a branch condition, and the like.

  For each input variable (for each input variable), its name, type, data format, etc. are registered as the input related information. The data format is the format (format) of the data of the input variable. Furthermore, an input condition (branch condition) that affects the output is also included.

  As is well known, the branch condition is a condition used for an instruction (for example, if-else syntax) for switching a code to be executed next depending on whether a certain condition is satisfied in the program. It is. For example, in the example shown in FIG. 3, there is a numerical value “50” as one of the branch conditions. For example, for example, the process to be executed varies depending on whether the value of the input variable InData is 50 or more. This corresponds to the case.

  Here, the data format indicates the data structure of the numerical value of the input variable and the like using a format name or the like. In the illustrated example, “Q12” is shown as an example.

  “Q12” is a fixed-point data format used when a decimal point is used in an integer variable. This is a data format for handling a numerical value including a decimal point, such as a Short type or a Long type, and the numerical value below Q differs depending on how many bits the value below the decimal point is expressed. As an example, FIG. 4 shows a bit value of Q12 format expressing “0.5”. This is “0x0800” in hexadecimal notation and “2048” in decimal notation.

The method for converting 0.5 to Q12 format is as follows.
0.5 × 2 ¹² = 0.5 × 4096 = 2048 = 0x0800
For reference, the method of converting 0.5 to Q15 format is as follows.

0.5 × 2 ¹⁵ = 0.5 × 32768 = 16384 = 0x4000
These conversion methods are well known.
The output related information includes an output method, a variable type, a data format, and the like. When the input information file 2 is generated, it is generated according to these variable types and data formats. Since the data format is Q12 in the illustrated example, the numerical data of the input information (input information file 2) is converted into the Q12 format as shown in FIG. Details will be described later.

  FIG. 5A shows a specific example of input conditions generated based on the specific example of the program specification information 1 shown in FIG. This is generated by the input condition determination unit 12 as described above.

  For example, the input condition determination unit 12 determines an input condition used for the test from the input related information (variable type, branch condition, etc.) of the program specification information 1. In the example of FIG. 3, two variables whose variable names are InFuncOn and InData are registered. InFuncOn has a variable type of Bool and no branch condition is defined. The variable type of InData is “Signed Long”, and branch conditions are defined ('50', '100', '150').

  First, InFuncOn is a Bool type variable as described above. As is well known, Bool type variables take two values, "True" and "False". Therefore, by registering the fact in advance, the input condition determination unit 12 sets the input condition corresponding to InFuncOn to “True” and “False” as shown in FIG.

  In InData, since the branch condition is defined as described above, the input condition determination unit 12 generates the input condition based on the branch condition. Here, as an example, by using boundary value analysis for the branch condition, the branch condition and the values before and after the branch condition are used as the input condition. That is, '50' and its preceding and following values '49' and '51', '100' and its preceding and following values '99' and '101', '150' and its preceding and following values '149' , '151' is an input condition.

  For example, as described above, the input condition determination unit 12 generates an input condition (such as an input value used for a test) for each input variable based on the variable type, the branch condition, and the like, for example, FIG. ) Is generated. From the above, the input condition is considered to be a data group such as “input value used for test”, which is determined for each input variable based on the variable type, the branch condition, and the like. May be.

  The test technique determination unit 13 determines an appropriate test technique to be applied to the program related to the program specification information 1 based on the input conditions generated by the input condition determination unit 12. Note that, as described above, here, description will be given using an example of selecting from among three types of test techniques, the round robin method, the all-pair method (pair-wise method), and the HAYST method, but the present invention is not limited to this example.

  The test technique determination unit 13 first calculates the “number of combinations” corresponding to the input conditions generated by the input condition determination unit 12 for each of the three types of test techniques. Here, FIG. 5B shows the calculation result of the “number of combinations” corresponding to the input conditions of the example shown in FIG. The “number of combinations” may be regarded as a number related to a combination of the data groups for each input variable. Alternatively, the “number of combinations” may be regarded as a number related to a combination of data groups (input conditions) for each variable.

  Here, as described above, if the combination condition is generated by the function of the existing tool, the “number of combinations” can be known. For example, in the round robin method, if a combination condition shown in FIG. 6 to be described later is generated, the “number of combinations” can be determined to be “18” as is apparent from FIG. Therefore, as an example, the “number of combinations” can be obtained for each test technique by such a method. However, the present invention is not limited to this example, and the “number of combinations” can be calculated without generating a combination condition. This will be described below.

  First, the round robin method is a method of testing all combinations, and the “number of combinations” can be calculated by multiplying all test conditions (such as the number of input conditions) of input variables. In the example of FIG. 5A, since there are two conditions (TRUE, FALSE) for the variable InFuncOn and nine conditions (49,... 151) for the variable InData, “number of combinations” = 2 × 9 = 18 (street ) If there is another input variable and there are three conditions, “number of combinations” = 3 × 2 × 9 = 54 (street).

  In the all-pair method (pair-wise method), the “number of combinations” can be calculated by multiplying the test conditions by using the top two input variables having a large number of input conditions among the input variables. In the example of FIG. 5A, since there are only two input variables (InFuncOn and InData), the number of combinations is inevitably multiplied by these two test conditions = 2 × 9 = 18. (Street) is calculated. That is, in this example, the number is the same as the round robin method.

  If there is another input variable and there are three conditions as described above, since the temporary input variable and the variable InData are the above two upper input variables, “number of combinations” = 3 × 9 = 27 (street) is calculated.

  In addition, in the HAYST method, first, as in the above-mentioned all-pair method, the above-mentioned process of “multiplying test conditions using the top two input variables having a large number of input conditions among input variables” is performed. A value obtained by raising the numerical value obtained by the above to a multiplier of 2 is defined as “number of combinations”. In the example of FIG. 5A, first, the numerical value is 2 × 9 = 18 as in the all-pair method. If this numerical value '18' is raised to a multiplier of 2, it will be '32'. Therefore, the “number of combinations” related to the HAYST method = 32 (streets).

  Since the multiplier of 2 is 2, 4, 8, 16, 32, 64,..., And so on, “18” is larger than “16”. For example, the numerical value “18” is compared in order from a value with a smaller multiplier of 2, and a multiplier of 2 (here, “32”) when it is first determined that the numerical value “18” is smaller. The numerical value '18' may be regarded as a value raised to a multiplier of 2. For example, the “number of combinations” in the HAYST method may be calculated by using, for example, the following calculation formula.

  Further, Sj is the number of conditions included in each parameter (as described above, there are 2 conditions and 9 conditions in the example of FIG. 5A). This is an expression for obtaining the minimum value of the number of combinations.

  Note that the all-pair method (pair-wise method) and the orthogonal table (HAYST method) are well-known test techniques used when performing a combination test. Known documents relating to the all-pair method (pair-wise method) are, for example, Patent Document 1 described above. Further, known literature relating to the HAYST method is, for example, the following references.

・ Reference: Software Test Symposium 2004 Tokyo Proceedings “Software test using orthogonal table-HAYST method”).
For example, as disclosed in Patent Document 1 above, with respect to the all-pair method, the number of test cases can be determined by generating test cases using an existing software tool such as PICT (Pairwise Independent Combinatorial tool). Is obtained as the “number of combinations”.

Similarly for the HAYST method, the above “number of combinations” can be obtained using existing software tools.
Of course, as described above, the “number of combinations” can be obtained without generating a test case.

  The test technique determination unit 13 determines a test technique (application test technique) to be used for the operation test of the program to be processed from each “number of combinations”. That is, here, a test technique having the smallest “number of combinations” among the three types is selected. However, in this example, the round robin method and the all-pair method have the smallest “number of combinations” and the same (= 18). In such a case, it is assumed here that a brute force method with a small amount of calculation is selected. In this regard, for example, it is assumed that information indicating which one to select when the number of combinations is the same is arbitrarily determined and registered in advance by a developer or the like.

  The “small amount of calculation” means that the amount of calculation for generating the combination condition is small. The amount of calculation for the combination condition generation process is determined by the test technique. In the “brute force method” in which the conditions are simply arranged, and in the “all pair method” in which a combination condition is generated by searching for a pair condition, the “brute force method” is used. It is clear that “computation amount” is smaller in the case of “”. Those skilled in the art can easily make such a determination, and based on this, for example, if the “number of combinations” is the same for the “brute force method” and the “all pair method” in advance, A program should be set up to adopt the “hit method”.

  The test technique application unit 14 generates a combination condition according to the applied test technique determined by the test technique determination unit 13. This is generated based on the determined test technique and the above input conditions. As already mentioned, the existing tool already has a function for generating a combination condition. Note that the combination condition may be regarded as synonymous with the test pattern, but in this description, input information (test pattern) is obtained by performing format conversion described later on the combination condition.

  Here, since the applied test technique is determined as “brute force method” in the above example, the combination condition shown in FIG. 6 is generated by generating the brute force combination of the above input conditions. Become. That is, as shown in the figure, a round-robin combination pattern of the above two types of data (“True”, “False”) and nine types of data (49, 50,..., 151), that is, 18 combinations. A pattern will be generated. Note that the details of this generation process are omitted here because the round-robin method, all-pair method (pair-wise method), and orthogonal table (HAYST method) are well known.

  The input information generation unit 15 generates input information (test pattern) such as the example shown in FIG. 7 based on the combination conditions (FIG. 6 and the like) generated by the test technique application unit 14. Here, the data format specified by the output related information of the program specification information 1 is applied to the combination condition generation result. In the example of FIG. 3, since the data format of the output related information is Q12, the input information shown in FIG. 7 is generated by converting the numerical data of the combination conditions shown in FIG. 6 into the Q12 format. .

  Since an example of the conversion method to the Q12 format has already been described, it is omitted here. For example, “49” in the numerical data of InData becomes “200704” in the Q12 format.

Finally, the generated input information is output as a file by the output unit 16, so that the input information file 2 is output to the outside.
As described above, according to the test support system 10 of this example, the input conditions used for the test are acquired from the specifications of the test target program, and the number of operation confirmations (number of combinations; number of test patterns) is minimized from the information. The test technique can be selected, the test technique can be automatically applied to the software test, and input information (test pattern) can be generated accordingly. That is, it is possible to automatically determine an appropriate test technique according to the program to be tested and generate a test pattern according to the test technique. It becomes possible to automatically determine the test technique used for the software test, and in particular, the effect of reducing the number of operation confirmations performed in the software test for the reason described above can be obtained.

  In addition, it is possible to generate input information (test pattern) used for the software test, and an efficient software test can be performed. Further, since the determination from the test technique to the generation of the input information is automatically performed, no human error occurs.

DESCRIPTION OF SYMBOLS 1 Program specification information 2 Input information file 10 Test support system 11 Input part 12 Input condition determination part 13 Test technique determination part 14 Test technique application part 15 Input information generation part 16 Output part

Claims (4)

An input condition generating means for generating an input condition based on the specification information of the target program;
Test technique determining means for determining the number of combinations corresponding to the input condition for each of the various test techniques registered in advance , and determining the test technique having the smallest number of combinations as the test technique to be applied to the target program;
Test pattern generation means for generating a test pattern of the target program according to the determined test technique;
A software test support system characterized by comprising: The specification information includes variable type, data format, and branch condition information for each variable of the target program.
The input condition generating means, each of each variable, the variable type or a software testing support system according to claim 1, wherein the generating the input condition is a data group which is determined on the basis of the branch condition. The software test support system according to claim 2 , wherein the number of combinations is a number related to a combination of the data groups for each of the variables. The test pattern generation means uses a combination of data groups for each of the variables as the test pattern, or a test pattern obtained by converting each numerical value in the combination according to the data format. Item 4. The software test support system according to Item 3 .