JP2000267888A - Method and instrument for measuring software test execution covering rate and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To quantitatively measure what degree a test is covered to a program module to be an object and to notify a person in charge of the test of it when a software test is performed. SOLUTION: Test point list data and test log data are inputted, and a log data label test point list collation processing part 12 decides whether or not a test point recorded in the test log data is registered with the test point list data. When it is registered, a test to the test point concerned is considered as executed and the contents of the test point list data are updated. The updated test point list data is finally referred to, a covering rate calculating part 16 calculates the rate of executed tests in test object points, and calculated results are outputted from a covering rate outputting part 17.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software test execution coverage measuring method, a software test execution coverage measuring apparatus, and a recording medium which are preferably used for accurately grasping the state of execution of a software test in the course of executing the software test. About.

[0002]

2. Description of the Related Art In a software development process, after a design and a manufacturing process, individual tests, combination tests, and comprehensive tests are performed, and the quality characteristics of software are confirmed in functional units through these tests. In particular, since the unit test is a test immediately after manufacturing, it plays a large role. In this unit test, since it is necessary to perform a test for each program module (subroutine, procedure, function, etc.), the number of test items is large, and accordingly, the number of test steps is also large.

Conventionally, extraction of test items in consideration of various input conditions for a program module to be subjected to a unit test has been performed manually. In addition, the person in charge of the test determined whether or not the test was actually performed for each test item. Then, the test execution manager collects the execution status of each test item reported by each test staff, and grasps the overall test execution status.

[0004]

As described above, in order to identify the test items performed and to grasp the overall implementation status,
It was necessary to coordinate the test implementation status with human intervention. For this reason, it was not possible to quickly ascertain the state of test execution. In addition, since the test points were manually extracted, it was necessary for a person to specify the processing route on which the test was actually performed based on the test execution result, and to perform the work of making the test on the processing route. For this reason, it was not possible to immediately obtain a quantitative value indicating how much the test was performed for all the test items from the test execution results.
As a result, the state of execution of the test is erroneously recognized, and a situation in which the test cannot be performed smoothly or a defect is found in a later process without sufficient testing has been caused.

Accordingly, the present invention has been made in view of such a problem, and in the process of performing a software test, the test execution status can be immediately grasped from the test execution result. It is an object of the present invention to provide a test execution coverage ratio measuring method, a test execution coverage ratio measuring apparatus, and a recording medium which can quantify and notify a test person.

[0006]

In order to achieve the above object, a software test execution coverage measuring method according to a first aspect of the present invention provides a method for measuring a program module as a result of a software test. A method for measuring whether or not the degree test has been covered, comprising: for each processing route derived from a processing branch point in a program module, inputting first data recording information for specifying the processing route; A step of inputting second data recording information specifying the processing route passed in the test for the module; and a processing route specified by the information recorded in the second data, Determine whether it matches any of the processing routes specified by the recorded information,
A storing step of storing information indicating that the test has been performed with respect to the matched processing route, and a plurality of processes specified by the information recorded in the first data with reference to the information stored in the storing step Calculating a value as an index indicating how much the test has been covered for the route.

A software test execution coverage measuring method according to a second invention is the software test execution coverage measurement method according to the first invention, wherein each of the program modules is configured in a hierarchical manner, and the value serving as the index is , Which are calculated in units of program modules of the same hierarchy.

According to a third aspect of the present invention, there is provided a software test execution coverage measuring apparatus which executes the software test execution coverage measuring method according to the first or second invention. A recording medium according to a fourth aspect of the present invention stores a program for executing the software test execution coverage measuring method according to the first or second aspect of the present invention.

[0009] This program is called FD or CD-RO.
The present invention can also be implemented by storing and distributing the program in a recording medium such as M, tape media, DVD-RAM, etc. (including the act of distributing and downloading this program via a network).

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings. 1. Configuration of embodiment 1.1. Functional Configuration FIG. 1 shows a test coverage percentage measuring apparatus 1 according to the present embodiment used for a software unit test. The test execution coverage measuring device 1 includes a test log data input unit 10 for inputting log data output as a result of a test on a program module, and a test log data analysis unit 11 for analyzing the test log data. A log data label / test point list comparison processing unit 12 for comparing and comparing a label described in the test log data with a test point list (both will be described later), and a test point list data input unit for inputting test point list data 13, a test point list data storage unit 14 for storing the test test point list data, a test point list data update processing unit 15 for updating the test point list data, and a coverage for calculating a test coverage for the program module. Rate calculating unit 16 and a coverage for outputting the calculated test execution coverage data It constructed an output unit 17.

The test log data input unit 10 receives test log data output from the program module test apparatus 3 shown in FIG. 2, and the contents of this data are analyzed by the test log data analysis unit 11, and the log data label is output. -Supplied to the test point list matching processing unit 12. On the other hand, test point list data output from the test point extraction device 2 shown in FIG. 2 is input to the test point list data input unit 13, and this data is stored and held in the test point list data storage unit 14.

The log data label / test point list collation processing unit 12 determines, according to the content analyzed by the test log data analysis unit 11, the processing route passed during the test indicated by the test log data, Is checked to see if it is described in. If it is described, the test point list data stored in the storage unit 14 is rewritten by the test point list data update processing unit 15. When the log data label / test point list comparison processing unit 12 completes the comparison for all the processing routes that have passed during the test, the coverage calculation unit 16 calculates the test execution coverage. Rate output unit 1
7 is output.

FIG. 2 shows a test point extracting device 2 for supplying test point list data to the test coverage measurement device 1 and a program module test device 3 for supplying test log data to the test coverage measurement device 1. ing. The test point extraction device 2 includes a program module input unit 20 for inputting a program module to be tested, a processing branch point labeling unit 21 for providing a label indicating a processing branch point to the program module by a process described later, A labeled program module storage unit 22 for storing the labeled program modules, a test point extraction unit 23 for extracting test points, and test point list data generation for outputting the extracted test points as list data An output unit 24;

From the program module input unit 20,
A program module to be tested is input. This program module is described in the programming language C ++ in order to perform design based on the object-oriented technology.

The processing branch point labeling unit 21 performs a process of labeling the program module input from the program module input unit 20. That is,
The syntax analyzing unit 21a analyzes a syntax described in the input program module to search for a processing branch point (for example, a portion indicated by a symbol P in FIG. 4B described later), and further searches for the branch point. A process of searching for a route derived from a point (a branch node; for example, portions indicated by reference numerals B1 and B2 in FIG. 4B described later) is executed. Further, the label generation / write unit 21b generates a label for specifying the branch node for each branch node, and executes a process of writing the label in the branch node. A label is given to a program module by the processing in the syntax analysis unit 21a and the label generation / write unit 21b.

The branch point label-attached program module storage section 22 stores and holds each program module to which the branch point label is assigned, and supplies the program module to the test point extraction section 23. When a test is actually performed on a program module, a program module with a branch point label to be tested is supplied to the program module test apparatus 3. The test point extracting unit 23 reads out the labeled program module from the branch point labeled program module storage unit 22, searches the label, and extracts the test point. The test point list generation / output unit 24 organizes and integrates the contents of the test points extracted by the test point extraction unit 23, collectively outputs test point list data, and outputs the data to the test execution coverage measuring device 1.

The program module test apparatus 3 includes a program module input unit 30 to be tested and a compiler 3
1, a debugger 32, and a test log data output unit 33. The labeled program module input from the test target program module input unit 30 receives a debug option and receives
Is compiled and linked, and an execution module is created. The tester activates this execution module using the debugger 32, sets a breakpoint and executes the program module one statement at a time, thereby performing a unit test while tracing the program module. In the course of this test, the label written in the program module to be tested is output to the log file every time the label passes through the processing branch point. This log file is output to the test execution coverage measuring device 1 by the test log data output unit 33.

1.2. Hardware Configuration FIG. 3 is a diagram schematically illustrating a hardware configuration of the test execution coverage measuring device 1, the test point extracting device 2, and the program module testing device 3. As shown in the figure, each of these devices is a CP that controls and controls the entire device.
U100, a RAM 101 on which a program for executing processing in each of the functional components (blocks denoted by reference numerals 10 to 17, 20 to 24, and 30 to 33) illustrated in FIGS. The ROM 102 in which the control program is stored and various data such as a labeled program module and test point list data are stored.
A hard disk drive (HDD) 103 and a communication control unit 104 serving as a connection interface between the devices while receiving data to be processed from an external terminal via a network. Connected to each other.

2. Operation of Embodiment As described above, the test point extraction device 2 generates test point list data (Process 1), and the program module test device 3 generates test log data (Process 2). The test execution coverage is input to the rate measuring device 1 and calculated (process 3). Therefore, the series of processes 1 to 3 will be described in order by dividing the items.

2.1. Test Point List Data Generation Processing (Process 1) FIG. 5 shows a flow of test point list data generation processing executed in the test point extraction device 2. First, a process of assigning a label to a test target program module is executed (S10).

FIG. 4A shows the format of this label. That is, this label is written in the branch of the processing in each program module, and the class name L1, the member function name (method name) L2, and "if,
A branch statement type L3 such as a “for, switch” statement, a function serial number L4 serving as an index in the function of interest, and “then, els
e, case "clauses and other clause types L5. The label information clarifies the position of the branch statement, that is, the position (processing route) in the program that gives the label. In other words, since the class name and member function name used in the function of interest are used as they are, just by looking at the label, it is possible to simply “see which class, which member function, which branch point, , To which branch node the label is attached ".

FIG. 4B shows an example in which labels are actually written in the branch nodes B1 and B2 derived from the branch point P in the program module. In the figure,
The line of the “printf” statement shown in, is the label. By embedding such a label in a program module, proc belongs to a class named Proc.
Then of serial number 1000 in the if statement in member function 1
This is output to the standard output by a printf statement when the debugger 32 is started and a unit test is executed, and this information is written to the above-described test log file by redirection.

FIG. 8 shows the contents of the label write processing for such a program module, which is executed in the processing branch point labeling section 21. In this label writing process, first, an initial value (for example, number 1000) of the label number (serial number L4 in the function in FIG. 4A)
Is set (S100). Next, the syntax analysis unit 21a reads the program module line by line (S1
01), it is determined whether there is a branch clause in the read character string (S102). When there is no branch node (S102;
NO), and it transfers to step S106. On the other hand, if there is a branch clause in the read character string, the label generation / write unit 2
1b generates the above-described label (S103), and performs a process of writing this label to the branch node of interest in the program module (S104). Then, the value of the label number is incremented by a predetermined number (S105), and it is determined whether or not the line in the source module read in step S101 is the last line (S106). If it is the last line (S106; YES), the labeling process for the program module ends. On the other hand, if it is not the last line (S106; NO), the pointer is moved to the next line (S107), and the processing from step S101 is executed again.

Returning to the description of FIG. The program module to which the label has been written is stored in the labeled program module storage unit 22. The test point extracting unit 23 reads the stored labeled program module, extracts test points as follows, and generates test point list data. First, the pointer is moved to the first branch section of the labeled program module (S
11), the label portion described in this branch clause (FIG. 4)
(B) The message in the printf statement indicated by, is acquired, and the contents are written to a file. FIG. 9B shows the contents of the test point list data.
(C). As shown in the figure, "Not yet" and "Done" in the left column indicate the test execution status,
The right column is the label (contents of the message output at execution)
It is.

Next, the fact that the execution status of the test point specified by the label is "not executed" is written in the file (S13). Then, it is determined whether the processing has been completed for all branch nodes (S14). When the processing has been completed for all branch nodes (S14; Y
In ES), the test point list data generation processing is completed. On the other hand, if the processing has not been completed for all the branch clauses (S14; NO), the pointer is moved to the next branch clause of the labeled program module (S15), and the processing from step S12 is executed again.

2.2. Test Log Data Generation Process (Process 2) As described above, the program module test apparatus 3 performs a test on a target program module,
Log data is recorded that records the test points whose passage was confirmed during the test. FIG. 6 is a flowchart showing the flow of the test log data generation process. First, when a source file including a labeled program module stored in the storage unit 22 is input from the test target program module input unit 30 (S20), the compiler 31
The source file is compiled and linked with a debug option to execute a process of generating an execution module (S21). Next, the debugger 32 is activated (S2
2), the execution module is activated on the debugger 32 (S23). Then, the tester sets a breakpoint, and tracing for the program module is started (S24). At this time, a message output when passing through the processing branch point is recorded in the log file (S25).

FIG. 9A shows the contents of the log file. As shown in the figure, during the debugging process, prin
The message output by the tf statement is output as a label corresponding to the processing route that has passed. When the trace using the debugger is completed, the test log data generation processing is completed. On the other hand, if the tracing has not been completed (S26; NO), the processing from step S24 is executed again.

2.3. Test Implementation Coverage Measurement Process (Process 3) When the test point list data and the test log data are prepared in this way, these data are input to the test execution coverage measuring device 1 and the test shown in FIG. The execution coverage measurement process is started. First, test point list data is input from the test point list data input unit 13 (S3).
0), test log data is input from the test log data input unit 10 (S31). The test log data analysis unit 11 analyzes the contents of the log data and extracts test points that have passed during the test, and the log data label / test point list matching processing unit 12 registers the test points in the test point list. Is determined.

That is, the pointer is moved to the first label of the test log data (S32), and it is searched whether or not the label of interest is registered in the test point list (S33). Then, it is determined whether or not there is a matching label in the test point list data (S34). When there is a matching label in the test point list data (S
34; YES), test point list data update processing unit 1
5 updates the contents of the test point list data stored in the test point list data storage unit 14. That is, a process of updating the execution status of the corresponding test point from “not performed” to “completed” is executed. FIG. 9B shows the status of this update processing in the test point list data.
(C). In FIG. 6B, since the test point which was set as “not performed” as the label B is recorded in the test log data of FIG. 7A, as shown in FIG. Done. "

As described above, when the test point list data is updated, it is determined whether or not the processing has been completed for all the labels in the test log data (S36). If the processing has not been completed for all the labels (S3
6; NO), move the pointer to the next label (S3
8) The processing of step S33 and subsequent steps is executed again. On the other hand, when the processing is completed for all the labels of the log data (S36; YES), the coverage ratio is calculated by the coverage ratio calculation unit 16, and the result is output from the coverage ratio output unit 17. Here, the number of test points that are “executed” in the test point list data after the test is performed is referred to as the number of passing points (number of tested points), and “not performed” in the test point list data before the test starts. The number of test points is calculated as the total number of points, and the coverage ratio = the number of passing points / total number of points Further, the coverage is calculated and output for each program module belonging to the same class.

3. Advantages of Embodiment (1) As described above, the value indicating the execution status of the software test is automatically measured without human intervention, so that the tester can grasp the execution status of the test at any time. Even a test for a simple program can be performed systematically based on the value of the test execution status. (2) In addition, since it is possible to quantitatively grasp the degree to which the test is performed on the software to be tested, it is possible to prevent erroneous recognition of the test execution status, and to carry out a defect in a subsequent process while the test is insufficient. Can be prevented, and a highly reliable test can be realized.

4. Modifications The present invention is not limited to the above embodiment, and various modifications are possible as follows, for example. (1) The coverage is calculated for each program module belonging to the same class. However, various criteria are set when calculating the coverage, such as the coverage for all source modules and the coverage for each source file. It is possible to obtain the coverage rate by using (2) Test point list data and test log data
Both had the same label format in which the class name, function name, etc. were described.Both of the data, such as the label consisting of the line number and the label adopted in the present embodiment, were used as the test target. Labels in different expression formats may be used as long as they can specify the processing route in the program module. (3) Although C ++ is targeted as a programming language, the present invention is also applicable to program modules described in various programming languages such as C and JAVA.

[0033]

As described above in detail, according to the present invention, when a software test is performed, the progress of the test is quantitatively indicated, so that the status of the test can be accurately grasped. In addition, the test staff does not have a false recognition of the test execution status, and the test can be performed smoothly.

[Brief description of the drawings]

FIG. 1 is a test coverage percentage measuring apparatus 1 according to the present embodiment.
FIG. 3 is a diagram showing a functional configuration of the embodiment.

FIG. 2 is a diagram showing a functional configuration of a test point extraction device 2 and a program module test device 3 according to the present embodiment.

FIG. 3 is a diagram showing an outline of a hardware configuration of each of the above-described devices 1 to 3.

FIGS. 4A and 4B show the contents of a label provided in the test point extracting apparatus 2, wherein FIG. 4A shows a label format, and FIG. 4B shows an example in which a label is embedded in a program module.

FIG. 5 is a flowchart showing a flow of a test point extraction process in the test point extraction device 2 according to the embodiment.

FIG. 6 is a flowchart showing a processing flow when a test log file is generated in the program module test apparatus 3;

FIG. 7 is a flowchart showing a flow of processing when measuring the test execution coverage in the test execution coverage measuring apparatus 1;

FIG. 8 is a flowchart showing a flow of labeling processing for a source module in the test point extracting apparatus 2;

FIG. 9 shows a test log file input to the test coverage coverage measuring apparatus 1 and contents of test point list data. (A) shows the contents of the log output at the time of debugging in the test execution device 3, and (b) and (c)
Is test point list data, and indicates that data indicating the execution status is changed before and after the test is executed.

[Explanation of symbols]

 Reference Signs List 1 Test coverage measurement device 2 Test point extraction device 3 Program module test device 10 Test log data input unit 11 Test log data analysis unit 12 Log data label / test point list matching processing unit 13 Test point list data input unit 14 Test point List data storage unit 15 Test point list data update processing unit 16 Coverage ratio calculation unit 17 Coverage ratio output unit 20 Program module input unit 21 Processing branch point label assignment unit 21a Syntax analysis unit 21b Label generation / write unit 22 Labeled program module Storage unit 23 Test point extraction unit 24 Test point list data generation / output unit 30 Test target program module input unit 31 Compiler 32 Debugger 33 Test log data output unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tadayuki Matsuya 1-9-1, Konan, Minato-ku, Tokyo Inside NTT Communicationware Co., Ltd. (72) Inventor Nobutaka Hattori Konan-ichi, Minato-ku, Tokyo (No. 9-1, NTT Communicationware Co., Ltd.) (72) Inventor Kingo Ando 1-9-1, Konan, Minato-ku, Tokyo F-term (inside, NTT Communicationware Co., Ltd.) Reference) 5B042 GA02 HH10 HH18 HH30 HH44 LA02 MA14 MC12

Claims (4)

[Claims] 1. A method for measuring the extent to which a test has been performed on a program module to be tested as a result of performing a software test, the process being derived from a branch point of processing in the program module. A step of inputting first data recording information for specifying the processing route for each route; and a step of inputting second data recording information for specifying the processing route passed in a test on the program module. Determining whether a processing route specified by the information recorded in the second data matches any of the processing routes specified by the information recorded in the first data; A storing step of storing information that the test has been performed on the route, and storing the information stored in the storing step. Calculating a value as an index indicating how much the test has been covered for a plurality of processing routes specified by the information recorded in the first data. Software test coverage rate measurement method. 2. The software test execution coverage measuring method according to claim 1, wherein each of the program modules is configured in a hierarchical manner, and the value serving as the index is calculated in units of program modules in the same hierarchy. A software test implementation coverage measurement method characterized by being performed. 3. A software test execution coverage measuring apparatus that executes the software test execution coverage measuring method according to claim 1 or 2. 4. A recording medium on which a program for executing the software test execution coverage measuring method according to claim 1 or 2 is recorded.