JP2000056961A - Software quality evaluating device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a software quality evaluating device capable of achieving highly precise/efficient quality evaluation and efficient/effective quality improvement by evaluating respective evaluation indexes by means of evaluation marks thereby giving priority to them in order of requiring the quality improvement. SOLUTION: The evaluation reference value of an evaluation item, a reference mark and an extraction/sorting condition are inputted. They are registered in an evaluation mark calculation reference file 8 and an extraction/sorting condition management file 9. An item density calculation part 3 calculates a check list number density and defect number density. An automatic itemized evaluation part 4 calculates the evaluation marks in accordance with evaluation mark calculation reference in the evaluation item units of cheek list item density, defect item density, a coverage rate, a scale, difficulty, the criticality of a defect and a defect analysis code. A comprehensive evaluation part 5 calculates the total of the evaluation marks of the respective evaluation items. Then, the outputs of the comprehensive evaluation part 5 are extracted/sorted and outputted in accordance with the registered extraction/sorting condition.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software quality evaluation technique, and more particularly, to a software quality evaluation apparatus for accurately and efficiently extracting weak points when software quality is quantitatively evaluated. Things.

[0002]

2. Description of the Related Art Conventional methods for quantitatively evaluating the quality of software include a method in which actual values of evaluation indices such as the number of checklists, the number of defectives, and the coverage ratio are set to a certain reference value (target value). In general, a computer is provided with a program for determining whether or not the time has been reached. However, in actual software development,
In many cases, sufficient time is not enough to improve the quality of each program so that the actual value of each index reaches the target value, and a lot of time is wasted on reviewing and reviewing review measures. Had to spend. As one method for solving such a problem, for example, Japanese Unexamined Patent Publication No.
As proposed in Japanese Patent Application Publication No. 146806, “Test work management support device”, by combining the test remaining evaluation based on the software reliability and the test sufficiency evaluation such as the coverage ratio, the improvement method is output for each combination pattern.
In some cases, the time required to consider countermeasures based on the evaluation results is reduced.

[0003]

In the conventional method proposed in the above-mentioned publication, an improvement method can be obtained by classifying each combination pattern. However, the weak points in one combination pattern are color-coded, so-called priority. It is not possible to evaluate and extract the parts that need to be improved. In other words, no matter how much combination patterns are refined, there are many cases where the program group concentrates on a specific pattern, and even after improvement, the pattern that is ultimately determined to be necessary is still the same. is there. The reason is that the standard value (target value) of each quality index is usually set in each system based on statistics and average values from past experience, but in most cases,
This is because some programs having very poor quality raise or lower the overall average value, and as a result, increase the variation of the average value or the like with respect to the reference value. As described above, since it is not clear which program / function should be reviewed with priority in the conventional quality evaluation method, there is a problem that it takes a lot of time and effort to review and review the review measures. there were. It is an object of the present invention to solve such problems of the conventional technology, quantitatively evaluate the quality of software, and aim for highly accurate and efficient quality evaluation and efficient and effective quality improvement work. It is to provide a possible software quality evaluation device.

[0004]

In order to achieve the above object, a software quality evaluation device of the present invention sets a reference value for each quality index that defines software quality, and scores a degree of achievement with respect to the reference value. Means for evaluating the degree to which quality improvement is necessary based on the total score of the quality index. The means for evaluating the degree of quality improvement is software evaluation means for evaluating the degree of quality improvement required for a plurality of software programs and prioritizing them in the order necessary for quality improvement. In addition, the quality index includes checklist item density, defect count density, coverage rate, software scale, software difficulty, percentage of highly significant defects in total defects, and qualitative defect analysis items. And the ratio of the number of analysis items specified in the above to the total number of analysis items, and the software evaluation means sets the reference value of the evaluation index regarded as important among the plurality of evaluation indices. There is provided a means for increasing the ratio of the evaluation index assigned higher priority to the total evaluation points, and scoring the priority so that quality improvement is required. Further, there is provided a means for extracting one or more evaluation indices from the plurality of evaluation indices, sorting and outputting the scores indicating the degree required for quality improvement by a given method such as an ascending order or a descending order.

More specifically, evaluation criteria for evaluation items
A data input unit for inputting (target) values, reference points, and extraction / sort conditions; a data registration unit for registering the input data in an evaluation point calculation reference file and an extraction / sort condition management file, respectively; An item density calculator that calculates the number of checklists and the number of defectives obtained by dividing the number of checklists and the number of defects in the quality data management file set in the program by the scale of the program, etc. Based on arbitrarily set quality control data and defect management data, an automatic evaluation unit for each evaluation item that calculates an evaluation score according to the evaluation score calculation criterion registered above, and a total of evaluation scores of each evaluation item are calculated. A comprehensive evaluation unit, an extraction / sort processing unit that extracts / sorts the output of the comprehensive evaluation unit according to the registered extraction / sort condition, ; And a data output unit for outputting a sorting result.

Further, the automatic evaluation unit for each evaluation item includes a quality data score calculation unit for calculating the achievement degree of the calculated item density and coverage ratio with respect to the reference value (target value) in accordance with the quality data evaluation score calculation standard, and a real data score calculation unit. A scale score calculator that compares the scale with the reference scale and calculates a score according to the scale evaluation score calculation standard; a difficulty score calculator that calculates a score according to the difficulty evaluation score calculation standard for each difficulty; It is characterized by comprising an importance score calculating section for performing score calculation according to the evaluation score calculation standard, and a failure analysis code score calculation section for performing score calculation according to the failure code evaluation score calculation standard for each failure analysis code.

[0007] Thus, the degree of achievement of each evaluation item of the checklist number density, the number of defective cases, and the coverage rate can be represented by a score in the evaluation unit, and sorting is performed according to the total points, so that the quality is improved in a descending order of the quality. Can be prioritized in the order required. In addition, priorities requiring quality improvement are given a score that takes into account the size of the software, the difficulty of the software, the percentage of the importance of defects, the percentage of defect analysis codes, and the reference score for each evaluation item is changed. By doing so, it is possible to prioritize the evaluation items (indexes) that the evaluator regards as important, and it becomes possible to extract programs and functions that require quality improvement with priority.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be specifically described below with reference to the drawings. FIG. 1 is a functional configuration diagram for explaining an embodiment of the software quality evaluation device of the present invention. As shown in the figure, the software quality evaluation device includes a data input unit 1 constituted by an input device (mouse, keyboard, etc.) of a computer for inputting evaluation score calculation criteria and extraction / sort conditions in the present invention. A data registration unit 2 for registering the inputted evaluation score calculation standard and the extraction / sort condition in the evaluation score calculation standard file 8 and the extraction / sort condition management file 9, respectively;
An item density calculator 3 for calculating a checklist number density and a defect number density obtained by dividing the number of checklists in the quality data management file 10 and the number of defects in the defect management file 11 arbitrarily set in advance by the scale of the program or the like. An evaluation item-based automatic evaluation unit 4 for calculating an evaluation score in accordance with the evaluation score calculation criterion registered above, based on quality management data and defect management data arbitrarily set in advance for various evaluation item units; Comprehensive evaluation section 5 for calculating the total score of each evaluation item from the calculation result of the automatic evaluation section for each item, and extraction / sort for extracting / sorting based on the processing result of the comprehensive evaluation section in accordance with the extraction / sort condition registered above. A processing unit 6;
And a data output unit 7 for outputting a result of the extraction / sorting process.

Here, the data input section 1 is constituted by a keyboard or other input device for inputting an operation instruction from an operator, the data output section 7 is constituted by a device such as a CRT display or a printer. The file 8, the extraction / sort condition management file 9, the quality data management file 10, and the defect management file 11 are realized in the form of a program or data on a storage device, a CPU, or a memory of a computer.

FIG. 2 shows a more detailed configuration of the automatic evaluation section 4 for each evaluation item and the evaluation score calculation reference file 8.
FIG. 6 is a diagram showing processing in an automatic evaluation unit 4 for each evaluation item.
As shown in the figure, the evaluation score calculation reference file 8 includes a quality data evaluation score calculation standard 801, a scale evaluation score calculation standard 802, a difficulty evaluation score calculation standard 803, a defect importance evaluation score calculation standard 804, and a defect analysis code. An evaluation score calculation standard 805 is stored.

The automatic evaluation unit 4 for each evaluation item includes a quality data score calculation unit 401, a scale score calculation unit 402, a difficulty score calculation unit 403, a defect importance score calculation unit 404, and a failure analysis code score calculation unit 405. It is configured. In the quality data point calculation unit 401, based on the checklist case density and the defect case density calculated by the item density calculation unit 3, and the coverage rate of the quality data management file 10,
The degree of achievement with respect to the evaluation criterion (target) value is calculated according to the quality data evaluation score calculation standard 801.

The scale score calculation section 402 compares the actual scale with the reference scale and calculates a score according to the scale evaluation score calculation standard 802. The difficulty score calculation unit 403 performs score calculation according to the difficulty evaluation score calculation standard 803 for each difficulty of the program. In the defect importance score calculation unit 404,
A score is calculated according to the defect importance evaluation score calculation standard 804 for each defect importance. Failure analysis code score calculation unit 4
In step 05, score calculation is performed according to the defect analysis code evaluation score calculation standard 805 for each defect analysis code.

FIG. 3 is a diagram showing a relationship between input data and a file at a storage destination. As input data, an evaluation reference (target) value of each evaluation item, a reference score, a score calculation formula, and the like are registered in the evaluation score calculation reference file 8, and an extraction key and a sort condition are stored in the extraction / sort management file 9. be registered. In addition, as a file arbitrarily set in advance, a quality data management file 10 including a program name, a scale, a difficulty level, the number of checklists, a coverage rate, and the like, a defect content such as a phenomenon, a cause, a countermeasure, and importance of a defect, There is a defect management file 11 including a defect analysis code and the like.

Hereinafter, the operation of this embodiment having such a configuration will be described in detail with reference to flowcharts. FIG.
It is a flowchart which shows the flow of a process of this software quality evaluation apparatus. First, input data is read from the data input unit 1 (step 101). If all the input data is not completed (step 102: N), the data registration unit 2
After registering the read input data in the file (step 103), the process returns to step 101 to read the next input data. When reading of all input data is completed (Step 102: Y), next, stored data is read from the data registration unit 2 (Step 104). Next, the item density calculation unit 3 calculates a checklist number density and a defect number density obtained by dividing the number of checklists in the quality management file 10 and the number of defects in the defect management file 11 by the size of the program (step). 10
5).

Next, the quality control data or the like arbitrarily set in advance for each evaluation item unit such as the number of checklists, the number of defectives, the coverage ratio and the scale, the difficulty, the importance of the defects, the defect analysis code, etc. On the basis of the defect management data, the respective score calculation units (401 to 405) execute a score calculation subroutine for each evaluation item for calculating a corresponding evaluation score according to the registered evaluation score calculation criteria (801 to 805) (step). 106). Next, the comprehensive evaluation section 5 calculates the total score of each of the calculated evaluation items and performs a comprehensive evaluation (step 107). Next, the extraction / sort processing unit 6 follows the extraction / sort conditions registered in the extraction / sort condition management file 9 (extraction conditions such as functional units and scales and sort conditions such as ascending order and descending order for the extracted items). Extraction / sort is performed (step 108). Finally,
The data output unit 7 extracts / sorts in the extraction / sort processing unit 6
The result of sorting (quality control table) is output (step 109).

FIG. 5 is a flowchart showing the flow of the above-mentioned score calculation subroutine for each evaluation item. First, the registered evaluation score calculation reference file 8 is read (step 201). If the quality data of the checklist count density, the defect count density, and the coverage rate is to be evaluated (step 202: Y), the quality data evaluation score calculation is performed. The standard 801 is read (step 203), and the quality data evaluation score is calculated (step 204). Next, when the scale evaluation is a target (Step 205: Y), the scale evaluation score calculation standard 802 is read (Step 206), and the scale evaluation score is calculated (Step 207). Next, when the difficulty evaluation is a target (Step 208: Y), the difficulty evaluation score calculation standard 803 is read (Step 209), and the difficulty evaluation score is calculated (Step 210). Next, when the defect importance or the defect analysis code is to be evaluated (step 21)
1: Y), the failure importance evaluation score calculation standard 804 and the failure analysis code evaluation score calculation standard 805 are read (step 212), and the failure analysis result and the like are stored in the failure management file 11.
(Step 213), and calculates the defect importance / defect analysis code evaluation score (Step 214).

FIG. 6 shows an example (a) of the above-mentioned quality data evaluation score calculation standard 801 and an example (b) of a calculation formula for calculating the score.
FIG. The example of the quality data evaluation score calculation criterion shown in FIG. 9A is an evaluation criterion (target value) of the checklist case density, defect count density, and coverage ratio, which are the evaluation items of the quality data, each of n1 cases / target values. ks, n2
Case / ks and n3% are set, and the respective reference points (points) are k1, k2, and k3. Here, “ks” indicates a kilo step, that is, 1000 steps.

In the example of the calculation formula in FIG. 2B, when the actual value of the target program has reached the evaluation reference value (target value), the evaluation score is the reference score of the holding point. On the other hand, when the actual value does not reach the evaluation criterion (target value), the evaluation score is a point obtained by dividing the actual value by the evaluation criterion value in consideration of the degree of achievement reaching the criterion (target). In this way, the achievement ratio of the actual value to the evaluation criterion (target) value is scored.

FIG. 7 is a diagram showing evaluation output examples of four types of programs when all the reference points in FIG. 6 are "1" (k1 = k2 = k3 = 1). In the example of FIG.
In the program 1, since the coverage rate, the checklist case density, and the defect case density all exceed the evaluation reference value (target value), the evaluation points are all “1”, and the total evaluation point is “3”. . For program 2, the evaluation score (0.3, 0.8, 0.8) is calculated in accordance with the actual values (20%, 80, 8) of the coverage rate, checklist density, and defective density. Calculated and total evaluation score "1.
Find the 9 "point. Similarly, for the program 3, the total evaluation score (0.8, 0.4, 0.5) corresponding to the actual value (60%, 0.4, 0.5) is "1.7". The points are the actual values (0%, 0, 0) for program 4.
The total evaluation score “0” of the total evaluation score “0” of the evaluation score (0, 0, 0) corresponding to (1) is obtained. The smaller the total evaluation score, the greater the demand for quality improvement. From the above,
It can be seen that the priority can be assigned among the four programs, and the quality needs to be improved in the order of program 4, program 3, program 2, and program 1.

FIG. 8 is a diagram showing an example of an evaluation output when the number of reference points shown in FIG. 6 is varied. This is based on the importance of the coverage rate among the checklist case density, the defect case density, and the coverage ratio, and the reference point (point) of the checklist case density and the defect case density is set to one point.
This is a case where the reference number of the coverage rate is set to two, and the one with a low coverage rate achievement level needs to have higher quality priority.

In this case, as in the case of FIG. 7, the total evaluation score of the program 1 is "4", the total evaluation score of the program 2 is "2.2", and the total evaluation score of the program 3 is "2.5". And the total evaluation score “0” of the program 4 is obtained. In this case, the order in which quality improvement is required is program 4, program 2, program 3, and program 1. By changing the reference score of each evaluation item in this way, it is possible to prioritize the evaluation items that the evaluator regards as important, and it is possible to extract a program or a function that requires quality improvement intensively.

FIG. 9 shows (a) a scale evaluation score calculation reference example,
It is a figure which shows the (b) difficulty evaluation score calculation reference example, and (c) defect importance and failure analysis code evaluation score calculation reference example. In the example of the scale evaluation score calculation standard in FIG. 9A, it is considered that a larger scale needs higher quality, and a difference from a reference value such as an average scale is reflected in the reference score. Further, in the difficulty evaluation score calculation reference example of FIG. 3B, it is considered that the higher the difficulty is, the more the quality needs to be improved, and the preset difficulty is reflected in the reference score. Further, in the defect importance and defect analysis code evaluation score calculation reference example of FIG. 3C, the defect importance is higher in the ratio of the number of defects having higher importance to the total number of defects, or the defect analysis code. If the number of failure analysis codes specified for (1) is high in the total number of failures, quality improvement is considered necessary, and the magnitude of the ratio is reflected in the reference score.

FIG. 10 is a diagram showing an example of an evaluation output in the case where specific actual values are substituted for all of these evaluation items and the quality improvement priority is comprehensively evaluated. In the specific example shown in the figure, the total evaluation score is “5” for program 1, “3.4” for program 2, and “4.2” for program 3.
The point of the program 4 is “1.5”, and it is understood that the order in which the quality improvement is required is the order of the program 4, the program 2, the program 3, and the program 1. The evaluation output from the comprehensive evaluation unit 5 is as follows. The extraction / sort processing unit 6 extracts / sorts items registered in the extraction / sort condition management file 9 according to extraction / sort conditions such as how to extract and sort (ascending / descending) items. Is performed, the data output unit 7 outputs the extraction / sort processing result.

The present invention is not limited to the above-described embodiment.
In addition to the time, the number of function matrices, and other evaluation indices that can quantitatively represent the quality of software, they can be newly set as evaluation items. The input data may be input manually from a keyboard or the like, or may be transferred from another data file to the apparatus of the present invention. The output format of each data, table, or the like in the data output unit may be a diagram or a graph in a format other than the table format described above. In the description of the above embodiment, the priority of quality improvement (evaluation of priority order) for a plurality of programs is described. However, it is possible to evaluate the degree of quality improvement for a single program. Needless to say.

As described above, according to this embodiment,
Automatic evaluation unit for each evaluation item based on the evaluation score calculation standard and comprehensive evaluation unit make it possible to objectively and quantitatively calculate each evaluation index of the software according to the degree of achievement with respect to the evaluation standard, resulting in poor quality The priorities are assigned in the order in which quality improvement is required, and as a result, highly accurate and efficient quality evaluation and efficient and effective quality improvement work can be achieved. Also, by changing the reference score of each evaluation item, it is possible to prioritize the evaluation items that the evaluator regards as important, and it is possible for the evaluator to extract programs and functions that judge that quality improvement is necessary. Become.

[0026]

According to the present invention, it is possible to objectively and quantitatively assign priorities in the order in which quality improvement is required by scoring each evaluation index, thereby achieving high accuracy and high efficiency. A software quality evaluation device capable of performing quality evaluation and efficient and effective quality improvement work can be realized.

[Brief description of the drawings]

FIG. 1 is a functional configuration diagram for explaining an embodiment of a software quality evaluation device of the present invention.

FIG. 2 is a diagram illustrating a more detailed configuration of an evaluation item-based automatic evaluation unit and an evaluation score calculation reference file, and a process performed by the evaluation item-based automatic evaluation unit.

FIG. 3 is a diagram illustrating a relationship between input data and a storage destination file.

FIG. 4 is a flowchart showing a flow of processing of the software quality evaluation device.

FIG. 5 is a flowchart illustrating the flow of a process of a score calculation subroutine for each evaluation item.

FIG. 6 is a diagram showing a quality data evaluation score calculation standard and a score calculation formula.

7 is a diagram illustrating an example of evaluation output of four types of programs when all reference points in FIG. 6 are “1” (k1 = k2 = k3 = 1).

8 is a diagram showing an example of an evaluation output when the number of reference points shown in FIG. 6 is changed.

FIG. 9 is a diagram showing an example of a scale evaluation score calculation reference example, a difficulty evaluation score calculation reference example, a defect importance level, and a failure analysis code evaluation score calculation reference example.

FIG. 10 substitutes specific actual values for evaluation items,
It is a figure showing an example of an evaluation output at the time of evaluating quality improvement priority comprehensively.

[Explanation of symbols]

 1: Data input unit 2: Data registration unit 3: Item density calculation unit 4: Automatic evaluation unit by evaluation item 5: Comprehensive evaluation unit 6: Extraction / sort processing unit 7: Data output unit 8: Evaluation score calculation reference file 9: Extraction / sort condition management file 10: Quality data management file 11: Defect management file 401: Quality data score calculation unit 402: Scale score calculation unit 403: Difficulty score calculation unit 404: Defect importance score calculation unit 405: Defect analysis code Score calculation unit 801: Quality data evaluation score calculation standard 802: Scale evaluation score calculation standard 803: Difficulty evaluation score calculation standard 804: Defect importance evaluation score calculation standard 805: Defect analysis code evaluation score calculation standard

Claims (5)

[Claims] 1. A means for setting a reference value for each quality index that defines software quality, scoring the degree of achievement with respect to the reference value, and evaluating the degree of quality improvement required based on the total score of the quality index. Software quality evaluation device characterized by having. 2. The means for evaluating the degree of need for quality improvement is software evaluation means for evaluating the degree of quality improvement required for a plurality of software and assigning priorities in the order necessary for quality improvement. The software quality evaluation device according to claim 1, wherein: 3. The quality index is a checklist count density, a defect count density, a coverage rate, a software scale, a software difficulty level, a ratio of a high-priority defect to a total defect, or a qualitative defect analysis. 3. The software quality evaluation device according to claim 1, wherein the number of analysis items specified in the items includes at least one of the ratios of the total number of analysis items. 4. The software evaluation means increases a reference value of an evaluation index regarded as important among a plurality of evaluation indices, and increases a ratio of evaluation indexes with higher priorities to an overall evaluation point, 4. The software quality evaluation device according to claim 2, further comprising: a means for scoring the priority so that the quality improvement is required. 5. The method according to claim 1, further comprising: selecting one of the plurality of evaluation indices.
5. The software quality evaluation apparatus according to claim 1, further comprising: means for extracting one or more evaluation indices, sorting and outputting the scores in a given method.