JPH05282178A - Software quality evaluating system - Google Patents

Abstract

PURPOSE:To simply judge whether the value of defective density is a proper value for a test advancing state or not by executing the evaluation of quality based upon relation between a test coverage ratio and the defective density. CONSTITUTION:The test coverage ratio 301 and the defective density 302 are respectively set up on a horizontal axis and a vertical axis and the data of defective density in respective modules constituting software are plotted as dots 305. A reference line 306 connected from an intersecting point between a line 304 of 100% test coverage ratio to be an evaluation reference and an average defective density value 303 of the whole system up to the position of the 0% test coverage is defective density proper to respective test coverage ratios in a test process. Thereby the quality of a module indicating a value close to the reference value 306 is stabilized and a module on the outside of a line 307 is regarded as abnormal quality. Thus relation between the test coverage ratio of respective test processes and defective density expressing the number of defects in a development size is found out and quality evaluation is executed based upon the relation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a quality evaluation method in a test process in software development, and more particularly to a quality evaluation method considering test sufficiency.

[0002]

2. Description of the Related Art Conventionally, as a software quality evaluation method, there is a method of evaluating quality by analyzing the number of defects and the contents of defects with respect to the development scale (total number of steps).
Further, Japanese Patent Laid-Open No. 6332/1987 and Japanese Patent Laid-Open No. 62-632
As disclosed in Japanese Patent No. 28845, there is an evaluation method by classification of defect contents by process, and an evaluation method by a satisfaction rate up to a predicted defect extraction target value.

[0003]

By the way, usually, in software development, the number of defects increases as the testing process progresses, and the total number of steps also changes as corrections are made.

However, in the above prior art,
In both cases, quality evaluation is performed by comparing the value of the number of defects during the progress of the test process with the target value at the time of completion of the test. Therefore, there is a problem in that the relationship between the degree of progress of the test process and the defect density that changes with the progress of the test cannot be known, and the individual evaluation of each element (program / module etc.) that constitutes the software cannot be known.

An object of the present invention is to provide a software quality evaluation method capable of judging whether or not the value of the defect density is an appropriate value for the test progress and evaluating each element constituting the software. Is to provide.

[0006]

In order to achieve the above object, the present invention obtains a relationship between a test coverage rate in a test process in software development and a defect density representing the number of defects with respect to a development scale. The quality is evaluated.

[0007]

For example, the test coverage rate at the module level is expressed by S0 (ratio of running modules to all modules) and S1 (call ratio to all inter-module calls). Therefore, according to the above means, the defect density (cases / Kstep = the number of defects per 1000 steps) is calculated from the total number of steps and the number of defects of each module, and the defect density and the module in which the defect density is calculated are calculated. Since the corresponding test coverage rate is calculated and the quality is evaluated based on these relationships, the relationship between the test progress status and the defect density becomes clear even if the total number of steps changes due to corrections. In addition, it is possible to accurately determine whether or not the value of the defect density is an appropriate value for the test progress, and it is possible to individually evaluate the quality of each module.

In this case, the instruction level or the program level can be evaluated similarly.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be specifically described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a system for realizing the quality evaluation method of the present invention.

In FIG. 1, 1 is a terminal device for registering defect information (cause of defect, name of defect module, defect phenomenon, etc.) extracted in a test process, and 2 is each module in a source file constituting software. The step number counting tool 3 for counting the number of steps of 3 is a test coverage rate measuring tool in the test process.
The tools described in JP-A-61-169944 can be used.

Reference numeral 4 is an information storage database for storing each of the above data (defect information, number of steps for each module, test coverage rate), 5 is an intermediate file for outputting raw data for creating a plot, and 6 is A graph creation tool for creating a plot drawing by inputting the contents of the intermediate file 5, and a printer 7 for outputting the created plot drawing.

In the above configuration, first, if a defect occurs in the test process, defect information (cause of defect, name of defective module, defective phenomenon, etc.) is input from the terminal device 1 and registered in the database 4. .. In parallel with this,
The step number counting tool 2 is used to count the number of steps for each module, and the test coverage rate measuring tool 3 is used to measure the test coverage rate in the test process for each module, and the respective count values and test coverage rates are measured. Are sequentially stored in the database 4.

When the series of tests are completed,
Defect information registered in database 4 (defect cause,
The name of the defective module, defective phenomenon, etc.) is read, and the number of defectives is counted for each module name unit in the information. Next, the number of steps and the test coverage rate corresponding to the counted module are read. The defect density is calculated from the read number of steps and the number of defects, and defect density data for each module is created. The created data is sequentially output to the intermediate file 5.

As a result, the data having the structure shown in FIG. 2 is stored in the intermediate file 5.

Therefore, in order to actually plot the contents of the intermediate file 5 created in this way into a plot, a general-purpose graph creating tool 6 is used, and the relationship between the defect density and the test coverage rate is shown in FIG. Create a plot as shown in. Then, the created plot diagram is output from the printer 7.

In FIG. 3, the test coverage rate 301 is plotted on the abscissa and the defect density 302 is set on the ordinate, and the defect density data of each module constituting the software is plotted at a point 305.

Here, a reference line 306 connecting from the intersection point of the line 304 of the test coverage rate of 100% and the average defect density value 303 of the entire system to the position of the test coverage rate of 0%, which is the evaluation standard, is defined in each test process. It is a reasonable defect density for the coverage rate.

Therefore, it can be said that the quality of the module showing a value close to the reference line 306 in FIG. 3 is stable. Further, when considering the entire system, the value of the defect density of each distributed module has a normal distribution with respect to the reference line 306, and the line 307 showing the value of the standard deviation can be drawn. The standard deviation value can be subtracted by the graphing tool performing statistical calculations. It can be said that the modules distributed more inside the upper and lower lines 307 drawn by the value of the standard deviation have more stable quality.

Conversely, the modules outside line 307 are said to be of abnormal quality.

Therefore, according to this embodiment, even if the total number of steps changes due to the correction, the quality becomes obvious at each module depending on the distribution position of the points 305. Further, it is possible to accurately determine whether the value of the defect density is an appropriate value for the test progress.

[0022]

As described above, according to the present invention, the relationship between the test coverage rate in the test process in software development and the defect density representing the number of defects with respect to the development scale is obtained, and the quality is evaluated based on this relationship. Therefore, even if the total number of steps changes due to the correction, it is possible to easily determine whether the value of the defect density is an appropriate value for the test progress. Moreover, there is an excellent effect that the quality of each element that constitutes the software can be evaluated.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a system for realizing the present invention.

FIG. 2 is a block diagram showing an example of data stored in a database.

FIG. 3 is an explanatory diagram showing an example of a quality evaluation plot diagram.

[Explanation of symbols]

1 ... Terminal device, 2 ... Step count tool, 3 ... Test coverage rate measuring tool, 4 ... database, 5 ... intermediate file, 6 ... graph creation tool, 7 ... printer, 3
01 ... Test coverage rate, 302 ... Defect density.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Masami Ishino 5030 Totsuka-cho, Totsuka-ku, Yokohama-shi, Kanagawa Pref., Hitachi Ltd. Software Development Division

Claims (1)

[Claims] 1. A software quality evaluation method characterized in that a relationship between a test coverage rate in a test process in software development and a defect density representing the number of defects with respect to a development scale is obtained, and the quality is evaluated based on this relationship.