CN117093504A - Software quality control method for defect measurement - Google Patents

Abstract

The invention belongs to the technical field of software quality control, and discloses a software quality control method for defect measurement. The method comprises the following steps: determining defect metrics including defect type and defect severity level; collecting defect data; defect statistics and analysis; calculating defect measurement indexes including defect density, defect removal rate, staged defect suppression rate, comprehensive test efficiency and comprehensive inspection efficiency; and (3) defect prevention and control, wherein the stage or process of finding defects or predicting possible defects is monitored in a key way according to the calculation results of defect statistics and analysis and defect measurement indexes, and different prevention and control measures are adopted according to different defect types and severity grades. The method can carry out defect control on each stage item of the software life cycle, and can effectively improve defect management efficiency and software quality.

Description

Software quality control method for defect measurement

Technical Field

The invention belongs to the technical field of software quality control, and particularly relates to a software quality control method for defect measurement.

Background

The software industry has taken an important place as a lead industry for realizing national economy and social informatization. In the software organization, quality, progress and cost are three major elements of the software project concerned, and the quality requirement of the software affects both cost and progress, and the three major elements are always first.

Software bugs, i.e., any problems, errors, or hidden functional bugs, flaws that disrupt the ability of normal operation that exist in a computer system or program. Software defect management is an important component of software quality assurance, and therefore, it is necessary to focus on software defect metrics during software development.

At present, a large number of specialized software defect management tools exist at home and abroad, wherein the software defect management tools are well known: rational Clear Quest, silk Central Issue Manger, track Record, etc., the functions provided by them each have an emphasis, and all can efficiently and conveniently Record and Track software defects, and the general functions thereof are as follows: defect information recording, defect flow control, defect state modification, defect information statistics, defect information inquiry, defect information output and the like. However, the existing software defect management tools work with defect states as driving forces, and the general management modes are still: the traditional mode of flow-oriented processing, namely finding defects, modifying defects and carrying out regression testing, is adopted. The defect management mode belongs to a passive mode, only passive measures are adopted to manage and process defects in software, and the defects are generally only modified, so that the information is not applied to future project development, the development level of a team is improved, the defect analysis information is not well applied to a software process, the quality of the software is improved, the workload of the software development is reduced, and meanwhile, certain disjoint exists between defect management and the software process, and the defect management information is not well applied to the software process.

Patent document CN106776316a discloses a defect prevention method for electric power information software, which inputs measurement information of each stage in the life cycle of the electric power information software and measurement information of the development process into an AODE prediction model, and calculates defect density of each stage; then searching a defect knowledge base of the power information software to obtain defect introduction reasons and corresponding preventive measures at each stage; and finally, performing targeted defect prevention, and reducing defect density. The defect prevention method for the power information software can provide a method and a technical foundation for developing defect prevention for power information system construction, can prevent defect introduction in early stage of power information system construction, reduce defect introduction rate, reduce repeated labor of defect introduction-test-modification, effectively increase pertinence of defect prevention, save development cost of information system projects, reduce failure risk of a software system and integrally improve overall construction quality of the power information software. However, the software defect prevention method of the patent only adopts one index of defect density to perform software defect measurement and prevention control of each stage, the measurement index is too single, the comprehensive and accurate defect measurement cannot be performed on each stage of the software development life cycle, further more effective software defect prevention and control measures cannot be provided, the software quality cannot be effectively ensured, and the quality and efficiency of software defect management are still to be further improved.

Disclosure of Invention

The invention aims to solve at least one technical problem existing in the background technology, and provides a software quality control method oriented to defect measurement, which is characterized in that firstly defect data are collected according to defect types and severity, then defect statistics and analysis and defect measurement index calculation work are carried out, finally defect prevention and control measures are formulated by referring to statistical analysis and measurement index calculation results, defect control is carried out on each stage item of a software life cycle, and defect management efficiency and software quality can be effectively improved.

In order to achieve the technical purpose, the invention adopts the following technical scheme:

a software quality control method for defect measurement, the method comprising the steps of:

step S1: determining defect metrics including defect type and defect severity level;

step S2: collecting defect data according to the defect type and the defect severity level;

step S3: performing defect statistics and analysis on the collected defect data according to defect metrics, including one or more combinations of the following statistics and analysis:

quantitative analysis of defects, defect summarization statistics, qualitative analysis of defects, analysis of defects before and after software release, missing detection analysis of software defects, standard recording of defect information and collection and processing of defect change;

Step S4: calculating defect measurement indexes including defect density, defect removal rate, staged defect suppression rate, full test efficiency and full inspection efficiency;

step S5: and (3) defect prevention and control, wherein the stage or process of finding defects or predicting possible defects is monitored in a key way according to the calculation results of defect statistics and analysis and defect measurement indexes, and different prevention and control measures are adopted according to different defect types and severity grades.

Further, in step S1, the defect types include a system defect, a data defect, a database defect, an interface defect, a function defect, and an interface defect.

Further, in step S2, the defect data sources include review data, test data, and user feedback data.

Further, in step S3, the defect quantitative analysis includes one or more of the following indexes: an index reflecting the quality of the product, an index reflecting the reliability of the product, an index reflecting the efficiency of defect discovery and repair, and an index reflecting the cost of defect repair;

wherein, the index reflecting the product quality includes:

defect density = number of defects/software scale;

Potential defect overview= (100% -defect removal rate before release)Defect density;

the indexes reflecting the reliability of the product include:

mean time to failure = software run-time/number of defects;

the indexes reflecting defect discovery and repair efficiency include:

defect detection rate = defect found at a certain stage/all defects belonging to that stage100%；

Pre-release defect removal rate = pre-release discovered defect/(pre-release discovered defect + software run last 3 months)100%；

Defect correction rate = number of defects that do not cause other problems during repair/total number of repaired defects100%；

The indexes reflecting the defect repair cost include:

average repair time =Defect repair time/number of defects;

average repair cost = average human cost of developerAverage repair time;

relative rework cost = rework workload/project total workload100% 。

Further, in step S3, the defect summary statistics include the following statistics: defect occurrence date, defect property, defect state, defect by product classification statistics, defect by reason classification statistics, defect test condition statistics and defect source statistics.

Further, in step S4:

the defect density is as follows:

DD=defects/KLOC

In the above formula, DD is defect density; defects are the number of defects found in a certain period of the software lifecycle; KLOC represents thousand lines of source code, kloc=sloc/1000, SLOC being the number of lines of source code;

the defect removal rate includes a periodic defect removal rate and an overall defect removal rate of the development process;

the periodic defect removal rate is:

or (b)

；

The overall defect removal rate of the development process is:

overall defect removal effectiveness of development process = total number of defects found by software development process/total number of defects;

the staged defect suppression rate is:

staged defect suppression rate = number of defects found at the stage/number of actual defects at the stage;

the overall test efficiency is as follows:

；

the overall inspection efficiency is as follows:

overall inspection efficiency = total number of defects found during the inspection phase/total number of defects found during each phase of the software lifecycle.

Further, in step S5, for the defects exceeding the metric index value in the defect metric, the following preventive and control measures are adopted for the demand stage, the design stage and the encoding stage, respectively:

the demand stage: listing a demand check table, and further executing a demand/test matrix to perform demand verification work, including: whether the function is complete, whether performance is considered, whether there is a fuzzy requirement, whether safety is considered, whether there is redundancy and error requirement, whether the requirement is too severe and whether the requirement is contradictory;

The design stage: through technical review test logic design, a process is mapped to an entity through establishing a process data matrix, and the life cycle of the data of the whole program is reflected;

encoding: unified coding specification, code review and unit testing are performed.

Further, the step S5 further includes a defect analysis operation, where the defect analysis operation includes: the defect distribution of the module, the cause distribution of the defects, the defect distribution by different discoverers, the defect distribution by different ways, the defect difference analysis, the defect distribution by time period and the Rayleigh analysis.

Further, before the defect prevention and control in step S5, defect prediction is further included, specifically:

predicting defects by using a prediction model estimator based on an improved algorithm of experience, and calculating the number of the predicted defects remained in the product;

the improved algorithm based on experience corrects the biased estimation quantity, and adopts the mean square error of the corrected relative deviation as the evaluation scale of the prediction effect.

Further, the mean square error of the corrected relative deviation is as follows:

in the above-mentioned method, the step of,mean square error, which is the corrected relative deviation; / >Is a mathematical expectation of relative deviation; />Is the variance of the relative deviation;xis a correction coefficient.

Compared with the prior art, the invention has the beneficial effects that:

(1) The invention provides a software quality control method for defect measurement based on the traditional defect management and software measurement technology, which comprises the steps of firstly collecting defect data according to defect types and severity, then carrying out defect statistics and analysis and defect measurement index calculation, finally referring to statistical analysis and measurement index calculation results, formulating defect prevention and control measures, and excavating more useful information for project management while carrying out defect management, thereby improving the efficiency of defect management, improving the monitoring strength of project progress conditions at each stage of software life cycle, and achieving the aim of guaranteeing and improving software quality;

(2) Compared with the patent document CN106776316A, the defect measurement-oriented software quality control method provided by the invention has the advantages that the defect measurement standard is preset, the defect data collection and the prevention and control measures are implemented according to the defect type and the defect severity level, meanwhile, during defect measurement, the multi-dimensional data statistics and analysis such as defect quantitative analysis, defect summarization statistics, defect qualitative analysis, defect analysis before and after software release, software defect missing test analysis, defect information specification recording, defect change collection and processing and the like are carried out, and a plurality of defect measurement indexes such as defect density, defect removal rate, staged defect suppression rate, comprehensive test efficiency and comprehensive inspection efficiency are calculated, so that the defect management is more comprehensive, objective, reliable and effective, and the acceleration of the software development process and the improvement of the software quality are facilitated;

(3) The invention provides a software quality control method for defect measurement, which further comprises a defect prediction step, wherein a review mode is adopted, a prediction model estimator is utilized, defects are predicted by an improved algorithm based on experience, the number of the defects remained in the product is calculated, and the mean square error of the corrected relative deviation is used as an evaluation scale, so that more effective decision reference basis is provided for subsequent defect prevention and control.

Drawings

FIG. 1 is a flow chart of a software quality control method for defect measurement according to embodiment 1 of the present invention;

FIG. 2 is a flow chart of a software quality control method for defect measurement according to embodiment 2 of the present invention;

fig. 3 is a diagram of a logistics center information management platform in accordance with embodiment 3 of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Example 1

The embodiment of the invention provides a software quality control method for defect measurement, as shown in fig. 1, comprising the following steps:

step S1: determining defect metrics

The defect metrics include defect type and defect severity level.

The defect types are shown in table 1, and include system defect, data defect, database defect, interface defect, functional defect and interface defect.

By defect classification, the problem of which type of defect is the greatest can be quickly found, and then the focus is on preventing and eliminating the type of defect.

The defect severity level includes: the major defects, minor defects and minor defects are specifically divided as shown in table 2 below.

By analyzing the severity level of the defect, the main and the sub-main parts can be arranged when the defect is eliminated, and the elimination time and the elimination progress are reasonably arranged. When the defect management data is used for reporting and processing, the severity level of the defect can be counted by fully utilizing the counting function, the system development quality is determined, and the progress is evaluated.

Step S2: collecting defect data

And collecting defect data according to the defect type and the defect severity level, and taking the defect data as a basis for later calculation of defect measurement indexes, defect statistics and analysis, defect prediction and defect prevention and control. The collection of defect data must be combined with a configuration management or change control system, and in particular, the CVS tool may be used to submit defect-related factors to form a defect database.

The defect data comes mainly from three aspects: review data, test data, and user feedback data.

Review data:

the reviewer reviews various documents generated in the system development, such as demand specifications, user manuals, system design documents, and the like. The reviewer finds defects through review, and for a relatively small-scale development team, the reviewer's job is served by the tester.

Test data:

the testers find defects through testing, and the testing is performed to check whether each function can be correctly realized according to the specific function requirements of the system description; and meanwhile, according to the internal working process requirement of the system, whether the system operates correctly according to the specification in the specification is checked.

User feedback data:

after delivery of the system to the user, the user may find problems in use, in part because of misuse and in part because of defects in system development. The user reports the defect in the form of written documents, telephones, emails, and Web forms.

Step S3: defect statistics and analysis

And carrying out defect statistics and analysis on the collected defect data according to the defect type and the defect severity level, and finding rules or trends from the defect statistics and analysis to be used as the basis for calculating defect measurement indexes, carrying out defect prediction and defect prevention and control.

The defect statistics and analysis specifically comprises: quantitative analysis of defects, statistics of defect summary, qualitative analysis of defects, analysis of defects before and after software release, missing test analysis of defects of software, standard record of defect information and collection and processing of defect change.

The statistics and analysis are described one by one.

Quantitative analysis of defects:

during defect analysis, quantitative analysis indexes are needed to be calculated, so that analysis results are measured for visual comparison.

The quantitative analysis indexes of the defects include the following items:

(1) index reflecting product quality:

defect density = number of defects/software scale;

potential defect overview= (100% -defect removal rate before release)Defect density;

(2) index reflecting product reliability:

mean time to failure = software run-time/number of defects;

(3) index reflecting defect discovery and repair efficiency:

defect detection rate = defect found at a certain stage at that timeAll defects trapped/belonging to this stage100%；

Pre-release defect removal rate = pre-release discovered defect/(pre-release discovered defect + software run last 3 months)100%；

Defect correction rate = number of defects that do not cause other problems during repair/total number of repaired defects 100%；

(4) An index reflecting the cost of defect repair:

average repair time =Defect repair time/number of defects;

average repair cost = average human cost of developerAverage repair time;

relative rework cost = rework workload/project total workload100%。

Defect summarizing and counting:

in defect analysis, a statistical method can be used for classifying and summarizing the collected defect changes, and the following specific statistics are carried out:

defect occurrence date statistics: according to the year and month statistics of change submission, analyzing and reflecting the dynamic trend of defect occurrence;

defect property statistics: the change property statistics are generally divided into defect change and demand change;

defect state distribution: the change state attribute is classified into a number of categories, but is not necessarily classified into a number of categories in the analysis, so 3 statistics are used: closing, suspending and processing; analyzing to mainly reflect the defect modification completion condition;

the defects are counted according to the product classification: the analysis can display the defect distribution condition of each software subsystem;

defect classification statistics by reason: classifying and counting according to the changed root cause attribute, wherein the statistics does not comprise the requirement change; the analysis reveals the distribution of defect causes;

and (3) defect test condition statistics: the root cause of the statistics related to the change is the defect change of system design, program coding, maintenance, external problems and the like; this analysis can expose problems with the software test itself;

Defect source statistics: the analysis mainly reflects the regional distribution of the user or the software agent, and discovers some client distribution rules.

The analysis statistics table and the graph can be summarized and counted according to a single attribute or a plurality of attributes.

Two examples of statistical analysis are given in tables 3 and 4 below.

Table 3 statistics of change causes table 1

Qualitative analysis of defects:

on the basis of quantitative analysis indexes of defects and summarizing statistical data, qualitative analysis is also required to be carried out on the defect condition of the software to obtain conclusion opinion. Analysis by the relevant personnel may be called upon for analysis and discussion if necessary.

For example, according to the program defect change test case statistics table in the above-mentioned additional table 5, the following conclusions can be obtained:

(1) up to 92% of the changes in defects are related to test scrutiny, with 22% of the changes belonging to obvious missed tests;

(2) wherein 44% of the changes are problematic in the regression testing procedure; after the defect program is modified in the maintenance stage, only the change is verified, and regression test is not performed, so that a new defect is caused;

(3) of which 8% have been found by testing prior to software release, but have not been addressed.

Based on the above analysis, targeted measures can be taken:

(1) Increasing the strength of the test. The problems found by the defects are added to the test cases;

(2) the software patch in the software maintenance stage can be released only by passing the regression test;

(3) when the software is released, the change which is not closed needs to be tightly controlled.

Defect analysis before and after software release:

the defect analysis before and after the software release comprises the defect analysis before the software release and the defect analysis after the software release, and the difference between the defect analysis before and the defect analysis after the software release is as follows:

(1) Variation of analysis

The object of the former analysis is a defect occurring in the software development stage, most often a defect of the software program code.

And analyzing defects after the software is released, wherein the analyzed objects are defects generated in the software operation stage. More problems of abnormality in software implementation, maintenance and user operation, confusion of data caused by unknown reasons, system errors caused by software version upgrading or various running environments, and new requirement changes proposed by users.

(2) The purpose of analysis is also different

The defect analysis before software release is mainly to evaluate the quality of the software in the development stage by calculating defect indexes, predict the running condition of the software after marketing and judge whether the software product can be released.

The defect analysis after the software release is to grasp the distribution and development trend of the defects through periodical analysis so as to control the occurrence of the defects and reduce the maintenance cost.

(3) Content of analysis is different

Most of the calculated metrics are suitable for pre-release defect analysis, e.g., the pre-release defect removal rate is designed specifically for it. However, classification statistics are applicable only to state distribution and classification statistics by reason.

For post-release defect analysis, most of the calculation indexes are not applicable. Only the average repair time index is important, which reflects the efficiency of handling defects during the maintenance phase. The classification statistics are basically suitable for post-release defect analysis, and are specially designed for post-release analysis, such as defect test condition statistics, defect source statistics and the like.

Software defect missing analysis:

by missing is meant that a defect in the software product is not found by the test group but is missing to the user, but is eventually found by the user. The earlier a defect is found in the software development process, the less cost it takes to resolve the defect. If the defect is found in the test set test and not when used by the user, the cost will be much less. If the defect is found by the developed group during the development process, the cost will be less.

Therefore, missing test analysis and pre-missing test are performed, so that defects are found as early as possible in the development process, and the cost of a software product is reduced and the quality of the software product is improved.

The purpose of performing the missing test analysis is to promote continued improvements in software quality and development testing procedures. Specifically, by analyzing the defects of missed detection in the development and test processes, corresponding preventive measures are formulated to avoid similar missed detection from occurring in future.

The general flow of missing test analysis is:

classifying missing tests;

carrying out statistical analysis on the missing detection classification data;

the identification and change of the whole process are carried out on the basis of statistical analysis;

on the basis of analyzing some special missed test items, some parts of the process are identified and changed;

the effect of the process change is illustrated using the metrology data.

For good results, it is preferable to periodically perform the missing test analysis in accordance with a schedule, which is a suitable frequency for one month.

It is difficult to perform a missing test analysis without the support of a defect tracking tool. Tools should be employed to maintain missed test defect data for all the different classifications. The Lotus Notes database can partition data in a variety of different ways, can create various views for the same batch of data, and can perform statistical analysis from various angles.

The statistics of the missing measurement classified data is to guide the improvement of the whole process, the statistical analysis firstly determines the frequency of the statistical analysis, and the statistical analysis is carried out in one quarter generally more properly. In the statistical analysis, it is necessary to compare the data of each classification item of a certain class with the data of all the other classification items of the class, and to perform such an operation for all the classes. Further subdivisions are made on classification items with a relatively large total number, and further statistical analysis work is performed.

Defect information specification record:

recording the discovered defects by adopting a defect record report; a defect fills out a defect entry table so that the defect can be better understood later.

Table 5 below is an example of a designed defect record table.

Collection and treatment of defect changes:

a change control library is established to store recorded defect change information, including all relevant information from the start of defect discovery to the creation of changes, until defect resolution, verification, and shutdown changes, all of which are valuable information required for defect analysis, throughout the life cycle of defect management.

The collection and processing of defect changes mainly comprises the following information items: the change number, the change theme, the date of change submission, the change state, the nature of the change, the date of change resolution, the root cause of change generation, the workload of solving the change, the workload of verifying the change, the severity registration of the change, the software product to which the change belongs, the subsystem of the change modification module, the stage of change generation, the source of change, the change test condition and the like.

Furthermore, for all stages of software development, the following defect analysis work is also required:

(1) The defect distribution of the modules is generally performed by using a bar graph or a pie chart, that is, the ratio of each functional module to find the bug, and the module with the largest bug is found to prove that more maintenance is required after the module is released.

In addition, the history data may refer to, for example, which module the last version finds the bug scale is a reference to this version. If a module finds that the proportion of bug drops significantly from the last version, it is likely that the module will require more testing.

(2) The distribution of the causes of defects, generally in the form of a bar graph or a pie graph, can be generally classified into structural defects, functional defects, usability defects, performance defects, security defects, demand defects, and interface text defects. Generally, if the architecture defect accounts for a large proportion, the design is described as having a great problem.

(3) According to the defect distribution of different discoverers, a histogram or a pie chart is generally used and is generally divided into a tester discovery, a developer discovery, a beta test discovery and an external client discovery. If the tester finds a bug below a certain proportion, the quality assurance test is proved to be insufficient.

(4) According to defect distribution in different modes, there are generally requirements for inspection, design test, code walk-through, JAD, manual test, automatic test and white box test. Generally, if the bug specific gravity found by JAD is low through the requirement examination, the design test and the code walk, the attention is not enough in the early stage of the test, and the contribution degree of the automatic test can be also indicated by the proportion between the manual test and the automatic test.

(5) Defect difference analysis is a found and solved curve relationship, and takes time as a horizontal axis, and the closer the defect difference analysis is to the defect difference analysis, the higher the product quality is.

(6) The defect distribution according to the time period is generally represented by a graph with time as a horizontal axis, mainly indicates at which stage the most bug is found, and has guiding significance for test summary.

(7) Rayleigh analysis, commonly known as zero defect tracking, generally involves a functional relationship (a complex mathematical function) of the total number of defects found up to a certain point in time and is used to infer how many more or less bugs are in the software after a number of days of testing, and how many more or less bugs are in the software after delivery to the user's hands.

Step S4: calculating defect metrics

The defect metric comprises: defect density, defect removal rate, periodic defect suppression rate, full test efficiency, and full inspection efficiency.

The defect metric index is the basis for performing defect measurement and control, and the purpose of calculating the defect metric index is to determine an index system for measuring defects and software quality.

To derive the calculation of each defect metric, a defect source/discovery matrix method is introduced herein that cross-classifies the defect data at various stages of discovering (removing) and implanting defects. This requires that for each defect found the defect source (i.e. the stage at which this defect is introduced) is determined.

Let j=1, 2, … k, be noted as each stage in the software lifecycle; let i=1, 2, … k be the kind of examination and test of different software lifecycle phases than the one comprising the maintenance phase (phase k).

The defect source/discovery matrix is shown in table 6 below.

Table 6 defect injection and removal matrix table at some stage in software development

In this matrix, only N _ij The cell (where i.gtoreq.j, i.e. the cell in the lower left triangle) has data. Diagonal unit (N) _ij Wherein i=j) represents a defect injected and detected at the same stage A number; the data in the cells below the diagonal line represent the number of defects that were injected early but detected later in the development. Because defects of late-development injection cannot be detected in early development, cells above the diagonal are empty. Boundary row N of matrix _i The number of defects excluded at this stage is indicated, while the boundary columns N. _j The number of defects originating at this stage is indicated.

The defect source/discovery matrix is triangular in shape because the defect source is typically at or before this stage, and the diagonal values of the test stage represent the number of bad repairs, a series of indicators of defect removal effectiveness can be obtained from this matrix.

The following is a method for calculating each defect metric according to the defect source/discovery matrix:

(1) Defect density

Defect density refers to the ratio of defect number defects found over a period of the software lifecycle to the system scale; the system scale is generally described in terms of the number of source code lines SLOC or the number of function points FP (Function Points), which are used herein to describe the system scale; the defect density DD is calculated as follows:

DD=defects/KLOC

DD is defect density; defects are the number of defects found in a certain period of the software lifecycle; KLOC represents thousand lines of source code, kloc=sloc/1000, SLOC being the number of lines of source code.

The defect density index reflects the intensity of defects in a software system and is one of important indexes for measuring the quality of the software.

(2) Defect removal rate

Defect removal rate: the defect removal rate may be a stepwise defect removal rate or an overall defect removal rate of the development process, reflecting the efficiency of finding or removing defects.

The periodic defect removal rate is calculated according to the following formula:

；

for the test phase, the number of defects implanted is typically small, and the defect removal efficiency for the test phase can be calculated by another method, namely:

；

the overall defect removal rate for the development process is calculated according to the following formula:

overall defect removal effectiveness of development process = total number of defects/total number of defects found by the software development process (including stages of development and testing).

(3) Rate of staged defect suppression

The staged defect suppression rate is similar to the defect removal rate, reflects the efficiency of the staged defect removed at a stage, and has the following calculation formula:

staged defect suppression rate = number of defects found at this stage/number of actual defects at this stage.

(4) Comprehensive test efficiency

The overall test efficiency belongs to one of the overall defect removal rates, reflects the efficiency of finding or eliminating defects in the test, and has the following calculation formula:

。

(5) Overall inspection efficiency

The overall inspection efficiency belongs to one of the overall defect removal rates, reflects the efficiency of inspection discovery or defect removal, and has the following calculation formula:

overall inspection efficiency = total number of defects found during the inspection phase/total number of defects found during each phase of the software lifecycle.

These defect metrics clearly indicate which stage in the development process should be addressed in order to improve software quality; the validity analysis may be performed on the entire project, or on a local area. The release and monitoring of these metrics longitudinally can be intuitive to the process capabilities of the developing institution. In addition, experience from previous releases also provides a basis for setting specific stage objectives and quality plans.

Step S5: defect prevention and control

According to the defect statistics and analysis and the defect measurement index calculation results, the stage or process of finding defects or predicting possible defects is monitored in a key way, and different preventive and control measures are adopted according to different defect types and severity grades so as to achieve the aims of controlling and eliminating the defects and reducing the number of the defects entering the next stage.

Aiming at defects exceeding a metric index value in defect measurement, the following preventive and control measures are adopted for a demand stage, a design stage and an encoding stage respectively:

The demand stage: and performing a good demand verification work. Several general items of verification are whether the functions are complete, whether performance is considered, whether there is a fuzzy requirement, whether safety is considered, whether there is redundancy and error requirements, whether the requirements are too severe, whether the requirements are contradictory, and the like. A commonly used method is to list a demand checklist and further execute a demand/test matrix;

the design stage: this stage is mainly through technical review of test logic design. The common practice of comparison is to build a process data matrix, i.e. CRUD matrix, map the process to an entity, and reflect the lifecycle (build, update, read, delete) of the data of the whole program;

encoding: the preventive measures at this stage mainly have unified coding specifications, code reviews and unit tests. The unified code specification is generally a unified requirement of a development manager, the code review is mutual review or review, and the most important is unit test, namely a general white box test.

Example 2

The method of the embodiment of the invention is further improved on the basis of the embodiment 1, and a defect prediction step is added before defect prevention and control.

As shown in fig. 2, the method includes:

step S1: determining a defect metric;

step S2: collecting defect data;

step S3: defect statistics and analysis;

step S4: calculating a defect measurement index;

step S5: predicting defects;

step S6: the defect prevention and control is to monitor the stage or process of finding or predicting the possible defect based on the defect measurement index calculation, defect classification and statistic analysis and defect prediction, so as to achieve the aim of controlling and eliminating the defect and reducing the defect number entering the next stage.

The following is mainly a detailed description of defect prediction in step S5 of adding improvement, and the rest of the steps are the same as in embodiment 1, and will not be repeated here.

Step S5: defect prediction, specifically:

predicting defects by using a prediction model estimator based on an improved algorithm of experience, and calculating the number of the predicted defects remained in the product;

the improved algorithm based on experience corrects the biased estimation quantity, and adopts the mean square error of the corrected relative deviation as the evaluation scale of the prediction effect.

Since the number of software residual defects is equal to the total number of software defects minus the number of found software defects, the problem of predicting the software residual defects can be converted into the problem of predicting the total number of software defects.

For estimators based on predictive models, we assume a total number of defectsThe estimated value is +.>Definition of relative deviation：

（1）

The mathematical expectation of the relative deviation is as close to 0 as possible, while the variance of the relative deviation is as small as possible, the higher the corresponding estimator prediction accuracy.

However, in many cases, a relatively small deviation is accompanied by a relatively large variance, or a relatively small variance is accompanied by a relatively large deviation, and therefore, a combination of these two factors is required.

The concept of mean square error in statistics, i.e. the mathematical expectation of the square of the deviation between the estimated quantity and the estimated parameter, is introduced hereIt is desirable to establish the "optimal" criterion of the estimator under the principle of both bias and discretization.

As previously mentioned, the estimators hereinRefers to the relative deviation->The mean square error of the relative deviation is: />

（2）

In the above-mentioned (2),is the relative deviation; />A metric that is an estimator; />Is the total number of defects; />Is->Is a function of the estimated value of (2); />Is the variance of the relative deviation; />Is a mathematical expectation of relative deviation; />、Is the mean square error of the relative deviation.

As can be seen from the above formula (2), the mean square error of the relative deviation is formed by summing two square terms of the variance and mathematical expectation, and the two terms of the left end and the right end of the formula (2) are relatively minimum at the same time, so that the requirements of deviation and discreteness are met. Thus, the mean square error of the relative deviation As the only scale of the evaluation model, +.>The smallest estimate is the optimal estimate.

Meanwhile, to reduce the estimated mean square error, the biased estimator is corrected.

Suppose after correctionThe following steps are:

（3）

（4）

（5）

mean square error：

（6）

Substituting formulas (4) and (5) into (6) to obtain:

（7）

in the above formulae (3) to (7),is the corrected relative deviation; />Is the relative deviation;xis a correction coefficient;is the total number of defects; />Is->Is a function of the estimated value of (2); />Is a mathematical expectation of relative deviation; />Mathematical expectation of corrected relative deviation; />Is the variance of the relative deviation; />Variance of the corrected relative deviation; />Is the mean square error of the corrected relative deviation.

From the binomial properties, it can be seen that in equation (7), the correction coefficientMean square error->Obtaining the minimum value: />。

Since the mathematical expectations and variances of the estimators are practically unavailable, they are calculatedxThe mean and variance of the samples may be considered for use.

Assume that there are k sets of historical data: wherein->Is the firstiTotal number of defects per sample->Estimated value of ∈10->The method comprises the steps of carrying out a first treatment on the surface of the By->The following sample mean and sample variance were obtained:

，（8）

（9）

and

（10）

in the above formulae (8) to (10),is the sample mean value; />Is the firstiThe relative deviation of the individual samples is such that, i=1，2，...，k；/>Is the sample variance; />Is the firstiRelative deviations of the individual samples.

In fact, the sample correction coefficients may be utilizedMultiplying any software defect prediction model by +.>Is a modification of (a).

For a less than 1Sometimes, the corrected estimated value is smaller than the number of found defects, and in this case, it is considered that all defects have been found, and the corrected estimated value is equal to the number of found defects.

When one development stage is finished, a review is made to see if the document or code of that stage meets the expected quality requirements, and based on this, it is determined whether the development can proceed to the next stage.

Review refers to 2 or more engineers reviewing another engineer's product, calculating the number of defects left in the product using a defect prediction algorithm, to find problems and activities of defects in the product.

Table 7 below is the predicted results for 4 panelists when evaluating:

/>

it can be seen that of the 7 non-history based estimators before improvement at 4 panelists, only the Mh (JK) and M0 (Chapman) estimators perform better; the Petersson's empirical defect estimation based on experiments and the Remus-Zilles model also perform better, in comparison the optimal non-historical data based estimator is Mh (JK). All models performed with much improvement, with the improved Mth (Chao) estimator and M0 (Lincoln-Peterson) estimator being the most optimal.

The effect of this improved algorithm for different panel populations is shown in table 8.

Table 8 optimal defect prediction algorithm

As can be seen from Table 8, when the number of panelists was 3 or 4 without using the history data, the Mh (JK) estimator was the best choice, and when the number of panelists was 5, the M0 (Chapman) estimator was the best choice, and when the number of panelists was 6, the Mt (MLE) was the best choice. After using the historical data and making improvements, the improved Mt (MLE) model was optimal for 3 panelists, and the improved Mth (Chao) and M0 (Lincoln-Peterson) models were selected for 4 or 5 panelists, with the improved M0 (Lincoln-Peterson) model being preferred for 6 panelists.

After the prediction of the residual defect number, the percentage of the defect number found in the review process to the total defect number in the product can be known, and the review can be concluded: either to let the development go to the next stage or to re-open the review.

Typically, if more than 90% of defects are not found in the review, a review will need to be re-conducted to find more defects.

Example 3

The embodiment of the invention applies the method in the embodiment 1 to the logistics center information management platform.

As shown in fig. 3, the logistics center information management platform in the embodiment of the invention is a logistics information management system which is applicable to the current Internet/Intranet-based network structure, has practical and easily-expanded functions according to the development characteristics of modern logistics.

The platform can manage the whole logistics business, realize information sharing and cooperative work of departments, warehouses and other network nodes of the logistics center, provide information support and decision management for each link, promote the whole logistics business of the logistics information function, and provide quick and timely logistics service for clients.

The system function targets are as follows:

1) The internet and the enterprise local area network can be used for collecting various information related to the logistics business, so that the real-time monitoring of the whole process of the logistics business is realized;

2) Each business process operation of the logistics center is effectively supported, coordination and consistency among all links of transportation, storage and distribution are ensured, and each work instruction is accurately and timely completed;

3) The system is suitable for the actual needs of a logistics center, can provide logistics information data and related statistical analysis, and divides functional modules according to the logistics business of modern enterprises, so that the system has good expansibility and maintainability.

The system function structure is shown in fig. 3, and the platform basic function consists of six functional modules, namely a basic information module, an operation management module, a transportation management module, a warehouse management module, a distribution management module and a system management module.

The operation management module, the transportation management module, the warehouse management module and the distribution management module are function modules formulated according to the business process of the logistics center, and the basic information module and the system management module are basic functions set for better showing the platform and the management platform. The functional modules are independent and have data exchange, so that the later management and maintenance are convenient.

Step S1: determining defect metrics

The defect metrics include defect type and defect severity level.

The defect types include system defects, data defects, database defects, interface defects, functional defects and interface defects, and are specifically shown in table 1 of example 1;

the defect severity level includes: the detailed divisions of the serious defect, the larger defect, the smaller defect and the slight defect are as shown in table 2 in example 1.

Step S2: collecting defect data

After the quality assurance team members find the defects, filling out a defect record table, wherein one defect is filled out a defect record table, and table 9 is a defect record example of the warehouse management module in the system.

Table 9 defect entry table example

Step S3: defect statistics and analysis

Defect statistics and analysis include defect type statistics and analysis and defect severity level statistics and analysis.

(1) Defect type statistics and analysis

Based on the division of defect types, the number of defects was counted for each sub-module in the logistics center information management system, as shown in table 10.

TABLE 10 number of defects for different defect type distributions

From the statistical analysis of table 10, it is clear that the number of defects of the transportation management module and the operation management module is the largest, and the defects are concentrated on the system defects, so that the system defects are main defects for developing the transportation management module and the operation management module, and the developer should pay special attention to this.

(2) Defect severity level statistics and analysis

In the development process of the logistics center information management system, the defect severity level is determined according to the defect condition of each sub-module, and the result is shown in table 11.

TABLE 11 distribution of sub-module defect severity levels

From the statistical results, the major defects and the minor defects of the transportation management module are still in a larger proportion, and the results are consistent with the defect type statistics above and must be brought to the attention of the responsible person of the development group. The development team responsible person knows through talking with the developer of this sub-module, and this part of developer does not carefully finish corresponding unit test according to the plan, makes this defect problem that discovers in the unit test stage leave over in the subsequent development stage, makes the defect that is not got rid of in time more difficult to get rid of, has caused the serious influence to development progress and development product quality.

Step S4: calculating a defect measurement index;

(1) Defect density

In order to calculate the defect density, firstly, the number of defects found in the system is counted, then the number of newly developed and repaired source code lines in the system is counted, and finally the defect density is calculated. And searching a version before the system is released and a version three months before the system is operated by using the CVS, and respectively counting the number of defects found by the system before the system is released and the number of defects found by the system 3 months before the system is operated. The defect number found in each stage before the system is released is 1053, the defect number found in the first three months of the system operation is 47, the total number of the system code lines is 27536 lines of source codes, and then:

defect density for system development = 1000 x 1053/27536 = 38.24 (defects/KLOC).

(2) Overall defect removal efficiency

In order to better ensure the quality of each stage of the project development, the quality team counts the defect number of each stage of the development in addition to counting the project defect data as a whole.

Table 12 is a matrix of defect source/discovery statistics for this entry. The number of defects generated and excluded at each stage of development can be intuitively observed in this matrix.

Table 12 matrix of defect source/findings statistics

The overall defect removal efficiency of the system can be represented by several indexes such as overall inspection efficiency, overall test efficiency and overall defect removal rate in the development process, and the indexes are respectively calculated as follows:

full audit efficiency IE:

overall test efficiency TE:

overall defect removal rate DRE of development process:

according to the calculation formula of the defect removal rate of each stage, the following calculation is respectively carried out:

high-level design inspection defect removal rate IE (I0):

low-level design inspection defect removal rate IE (I1):

code inspection defect removal rate IE (I2):

cell test defect removal rate TE (UT):

component test defect removal TE (CT):

system test defect removal rate TE (ST):

for the test phase, since defect injection is typically small, the defect removal rate index for the test phase is calculated as follows:

therefore, the defect removal rate at each test stage is calculated as follows:

cell test defect removal rate TE (UT):

component test defect removal TE (CT):

system test defect removal rate TE (ST):

analysis of the defect index calculation results shows that the overall test defect removal rate and the overall defect removal rate index of the development process are higher, so that the expected target of quality control is achieved, but the overall inspection efficiency and the defect removal efficiency at certain stages are lower, and the expected value is not reached, so that the improvement on the later process improvement and defect prevention and control measures is required.

Since defects are found to be easier to remove the earlier, the efficiency of inspection and testing work can be re-enhanced to improve the quality of later versions of software, minimizing the number of defects left in the released version of software.

Step S5: defect prevention and control

Based on the defect measurement index calculation and the defect statistics and analysis results, the following measures are formulated for the measurement process improvement of the organization so as to promote the defect measurement process improvement and more effectively prevent and control the defects:

(1) The proportion of the larger defects and the serious defects found in the system reaches 46%, which indicates that the number of defects with higher severity level in the system is more, and if the defects cannot be eliminated in time, the defects possibly have larger influence on the system performance, so that the technical scheme audit and the code coding quality control are enhanced, the design scheme is demonstrated in detail, and the excessive proportion of the serious defects is avoided.

(2) The system defects and the interface defects respectively reach 39% and 19%, and occupy larger proportion in the whole defect types, which is the two most occurring defects in all types, so that in the improvement of the software process, the influence of system factors such as an operating system, a database, application business and the like on the software quality should be comprehensively considered, and the system defects are reduced; meanwhile, quality control is carried out on the interface design, so that the probability of occurrence of interface defects is reduced.

(3) The comprehensive inspection efficiency of the system is 57%, the efficiency is low, the strength and depth of the inspection work are required to be enhanced, the inspection of the requirement, the overall design and the bottom design are required to be enhanced, and all measures of the combined inspection, verification and confirmation process are enhanced, so that defects are discovered and eliminated in early stage as much as possible.

(4) The defect removal rate of the low-level design inspection is only 23%, and a significant portion of the defects at the stage entrance and the defects injected are not found in the inspection at this stage, so that the inspection and verification work of the low-level design should be enhanced to improve the defect removal effectiveness at this stage.

(5) The code inspection defect removal rate was 49%, although this stage excludes a larger number of defects (368), the number of defects that were not excluded from the stage itself reached 355, i.e., the code inspection missed a large number of defects that occurred in the code stage, which were not discovered until the test stage, which was detrimental to early detection and elimination of defects. Thus, code walkthrough and code review should be enhanced, and the code itself should be reviewed.

The foregoing description is only exemplary of the invention and is not intended to limit the invention. Any modification, equivalent replacement, improvement, etc. made within the scope of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A software quality control method for defect measurement, the method comprising the steps of:

step S1: determining defect metrics including defect type and defect severity level;

step S2: collecting defect data according to the defect type and the defect severity level;

step S3: performing defect statistics and analysis on the collected defect data according to defect metrics, including one or more combinations of the following statistics and analysis:

quantitative analysis of defects, defect summarization statistics, qualitative analysis of defects, analysis of defects before and after software release, missing detection analysis of software defects, standard recording of defect information and collection and processing of defect change;

step S4: calculating defect measurement indexes including defect density, defect removal rate, staged defect suppression rate, full test efficiency and full inspection efficiency;

step S5: and (3) defect prevention and control, wherein the stage or process of finding defects or predicting possible defects is monitored in a key way according to the calculation results of defect statistics and analysis and defect measurement indexes, and different prevention and control measures are adopted according to different defect types and severity grades.

2. The method according to claim 1, wherein in step S1, the defect types include a system defect, a data defect, a database defect, an interface defect, a function defect, and an interface defect.

3. The method of claim 1, wherein in step S2, the defect data sources include review data, test data, and user feedback data.

4. The method according to claim 1, wherein in step S3, the defect quantification comprises one or more of the following indicators: an index reflecting the quality of the product, an index reflecting the reliability of the product, an index reflecting the efficiency of defect discovery and repair, and an index reflecting the cost of defect repair;

wherein, the index reflecting the product quality includes:

defect density = number of defects/software scale;

potential defect overview= (100% -defect removal rate before release)Defect density;

the indexes reflecting the reliability of the product include:

mean time to failure = software run-time/number of defects;

the indexes reflecting defect discovery and repair efficiency include:

defect detection rate = defect found at a certain stage/all defects belonging to that stage100%；

Pre-release defect removal rate = pre-release discovered defect/(pre-release discovered defect + software run last 3 months) 100%；

Defect correction rate = number of defects that do not cause other problems during repair/total number of repaired defects100%；

The indexes reflecting the defect repair cost include:

average repair time =Defect repair time/number of defects;

average repair cost = average human cost of developerAverage repair time;

relative rework cost = rework workload/project total workload100% 。

5. The method according to claim 4, wherein in step S3, the defect summary statistics include the following statistics: defect occurrence date, defect property, defect state, defect by product classification statistics, defect by reason classification statistics, defect test condition statistics and defect source statistics.

6. The method according to claim 1, characterized in that in step S4:

the defect density is as follows:

DD=defects/KLOC

in the above formula, DD is defect density; defects are the number of defects found in a certain period of the software lifecycle; KLOC represents thousand lines of source code, kloc=sloc/1000, SLOC being the number of lines of source code;

the defect removal rate includes a periodic defect removal rate and an overall defect removal rate of the development process;

the periodic defect removal rate is:

or (b)

；

The overall defect removal rate of the development process is:

Overall defect removal effectiveness of development process = total number of defects found by software development process/total number of defects;

the staged defect suppression rate is:

staged defect suppression rate = number of defects found at the stage/number of actual defects at the stage;

the overall test efficiency is as follows:

；

the overall inspection efficiency is as follows:

overall inspection efficiency = total number of defects found during the inspection phase/total number of defects found during each phase of the software lifecycle.

7. The method according to claim 1, wherein the step S3 further includes a defect analysis operation, the defect analysis operation including: the defect distribution of the module, the cause distribution of the defects, the defect distribution by different discoverers, the defect distribution by different ways, the defect difference analysis, the defect distribution by time period and the Rayleigh analysis.

8. The method according to claim 1, wherein in step S5, for defects exceeding the metric index value in the defect metric, the following precaution and control measures are adopted for the demand phase, the design phase and the encoding phase, respectively:

the demand stage: listing a demand check list, and further executing a demand test matrix to perform demand verification work, including: whether the function is complete, whether performance is considered, whether there is a fuzzy requirement, whether safety is considered, whether there is redundancy and error requirement, whether the requirement is too severe and whether the requirement is contradictory;

The design stage: through technical review test logic design, a process is mapped to an entity through establishing a process data matrix, and the life cycle of the data of the whole program is reflected;

encoding: unified coding specification, code review and unit testing are performed.

9. The method according to any one of claims 1-8, further comprising defect prediction, in particular, before performing defect prevention and control in step S5:

predicting defects by using a prediction model estimator based on an improved algorithm of experience, and calculating the number of the predicted defects remained in the product;

the improved algorithm based on experience corrects the biased estimation quantity, and adopts the mean square error of the corrected relative deviation as the evaluation scale of the prediction effect.

10. The method of claim 9, wherein the mean square error of the corrected relative deviation is as follows:

in the above-mentioned method, the step of,mean square error as corrected relative deviation；/>Is a mathematical expectation of relative deviation; />Is the variance of the relative deviation;xis a correction coefficient.