CN108021498A

CN108021498A - A kind of test job amount distribution method based on software reliability prediction

Info

Publication number: CN108021498A
Application number: CN201610977069.9A
Authority: CN
Inventors: 周毓明; 冯义洋; 卢红敏; 徐宝文
Original assignee: Nanjing University
Current assignee: Nanjing University
Priority date: 2016-11-04
Filing date: 2016-11-04
Publication date: 2018-05-11

Abstract

The present invention provides a kind of test job amount distribution method based on software reliability prediction, comprises the following steps：1) collection and processing of software data collection；2) structure of software defect prediction model；3) software defect finds the structure of model；4) software defect finds model V₀Release parameter is estimated；5) software defect finds model V₁Release parameter is estimated；6) software V₁The optimal test job amount distribution of version.The present invention is a kind of test job amount allocative decision, solve the problems, such as at present how greatest benefit in the case that test resource is limited defect in testing out system.The program takes full advantage of the information of prior software release, is allocated come the test job amount of the system to current version, so that the accumulation defect of most numbers may finally be found.

Description

Test workload distribution method based on software reliability growth model

Technical Field

The invention belongs to the technical field of software testing, and particularly relates to a test workload distribution method based on a software reliability growth model.

Background

Software testing is an exploratory activity aimed at helping software practitioners assess the quality status of software under test. In the complete software development process, the software testing activities all the time, which occupies most of the time of the software development project, require a large amount of human and material resource investment. To ensure the quality of the software, many techniques have been used to predict which modules are prone to defects, most of which are to select software modules for testing based on their probability of defects, their expected number of defects, or their density, but for the final goal: the subject of reducing the test workload to improve the software quality has been studied little.

The distribution of the test workload of each module in the software is always an important work in the software test process, under the condition that the test resources are limited, if the modules are easy to generate defects can be effectively identified in advance, the distribution of the test workload can be greatly helped, the test workload can be influenced by adopting which test workload distribution strategy in the actual industrial production, and even a poor resource distribution strategy can lead to the fact that the test workload is greatly increased. The traditional software testing workload distribution scheme has the following characteristics: 1. the distribution scheme of the test workload is less; 2. the specific criteria for selecting the module to be tested is not clear; 3. there is a lack of evaluation criteria for the test workload distribution strategy. The traditional software testing workload distribution method can not meet the actual requirement more and more, and a testing workload distribution scheme is urgently needed, so that the testing workload of the system of the current version can be distributed by fully utilizing the information of the software of the previous version under the condition of limited testing resources, and the maximum number of accumulated defects can be found finally.

Disclosure of Invention

The invention aims to provide a test workload distribution method based on a software reliability growth model, which solves the problem of how to test the defects in a system with the maximum benefit under the condition of limited test resources at present, and further greatly improves the software test work efficiency, thereby better controlling the quality of products.

In order to achieve the above object, the present invention provides a method for distributing test workload based on a software reliability growth model. The method comprises the following steps:

1) acquisition and processing of software data sets by means of defect patch reporting, software version V₀The data information to be collected includes: collecting defect information, collecting and processing test cases and collecting basic measurement; software version V₁The data information to be collected includes: basic metric collection, total test workload to be distributed, etc.

2) Constructing a software defect prediction model, namely after the step 1), converting V into V₀And V₁The measurement information of each module is organized into a form of a feature vector, the defect number is used as a target variable, and a defect prediction model (pair V) is established by prediction technologies such as random forest, linear regression, support vector machine and the like₀Set up training and for V₁Making a prediction);

3) constructing a software defect discovery model; the model can estimate the number of the discoverable defects relative to given test resources, and the model used by us is an expanded exponential reliability growth model, also called Goel-Okumoto model (Go model for short), because the model is the simplest NHPP model and has a constant defect detection rate in any test time, which means that the parameter estimation of the model is much easier than that of other SRGM; the model is used to represent the relationship between the test workload and the detected accumulated defects, and the calculation formula is as follows:

in the formulaFor the module m given a certain amount of test workload_iNumber of defects, t, that can be found finally_iIs a module m_iAssigned test workload, b_iNumber of defects that can be found for a unit test workload, a_iIs a module m_iInitial defect number of S_iIs a module m_iSize of (a) b₀Is a constant.

4) Software defect discovery model V₀Estimating version parameters; version V₀Parameter a of_iAnd b₀It needs to be estimated from the existing data, and the calculation formula is as follows:

wherein H_iIs a module m_iThe number of actually detected defects of the optical disc,is a module m_iThe number of new/modified code lines of (c),is a module m_iThe number of code lines multiplexed in (1). R₁And R₂Is the residual defect rate, which indicates that R still remains in the 1KLOC line in the new/modified code, respectively₁A defect; in the multiplexed code, R still remains in the 1KLOC line₂And (4) a defect.

Followed by estimation b₀It indicates the number of defects detectable per unit of test workload, derived from the actual assigned test workload and the number of defects detected, and is formulated as follows:

H_total＝a_total*[1-exp(-(b₀/S_total)*t_total)](4.2)

thus, there are:

b₀＝-S_total/t_total*log(1-H_total/a_total) (4.3)

wherein S_totalIs the total size of all modules, t_totalIs the total actual test workload, H_totalIs the total number of defects actually detected, and has

5) Software defect discovery model V₁Estimating version parameters; due to V₁The actual defect number of the version is unknown, so the method in step 4) cannot be used to estimate V₁Parameters in versions, we adopt the method of passing through previous version V₀To estimate V_iOf (2), in particular, V₁B of (a)₀And V₀Taking the same value, V₁Module m in version_iInitial defect number a of_iThe estimation method is as follows:

case 1: if m_iAt V₀In existence of

A. If it is notSuppose V₀Version m_iHas k defects at V_iIs repaired, then:

B. if it is notDescription of m_iPractically no defects, so we can show a decrease in V₁Version m_iA of (a)_iThe value:

a_i(v_i，m_i)＝a_i(v_i-1，m_i)/10 (5.2)

case 2：m_iat V₀Is absent in (V)₁The newly added module. The calculation is as follows:

wherein model is at V₀The defect prediction model, Aplymodel (model, m), established above_i) M predicted for applying the defect prediction model_iInstead of the number of defects actually detected.

6) Software V₁Distributing the workload of the version optimal test; known as V₁Each module m in the edition_iA of (a)_iAnd b_iValue and test workload t to be allocated_totalThen, the optimal test workload distribution strategy of the version under the Go model can be obtained, and the optimal test workload distribution strategy is distributed to m according to the strategy_iThe corresponding workload is substituted into the defect discovery model, and finally the maximum number of accumulated defects can be discovered. The calculation formula is as follows:

A_i＝a_ib_i(i＝1，2，...，M) (6.1)

A₁≥A₂≥...≥A_k-1≥A_k≥...≥A_M(6.2)

next, we can start from { λ₁，λ₂，...，λ_MGet the best parameter lambda in }^*：

(1) Setting k to 1;

(2) from equation 6.3, λ is calculated_k；

(3) If A is_k≥λ_k＞＝A_k+1Then λ^*＝λ_k(stop) otherwise, set k-k +1 and go back to step (2) finally we can calculateThe following were used:

wherein,is allocated to module m_iThe final test workload of (a);

further, the specific steps of the step 1) are as follows:

step 1) -1: an initial state;

step 1) -2: to V₀Collecting the defect information, and counting the actual defect number of each module;

step 1) -3: to V₀The test cases are processed, and V is counted₀Actual test workload of; for V₀Manually ensuring that there is a test case that can trigger the defect; specifically, if a test case which can trigger a certain defect cannot be found for the defect, the defect is excluded from statistics;

step 1) -4: statistics V₀Code metric and process metric information for each module of (a);

step 1) -5: statistics V₁Code metric and process metric information for each module of (1), initialization V₁Is measured by the available test workload value.

Step 1) -6: and finishing the acquisition and processing of the software data set.

Further, the specific steps of the step 2) are as follows:

step 2) -1: an initial state;

step 2) -2: will V₀And V₁The measurement information of each module is organized into a form of a feature vector, and the defect number is used as a target variable;

step 2) -3: with V₀Establishing a prediction model for the training set;

step 2) -4: with V₁For test set, pair V₁Each module in the set outputs a defect prediction value;

step 2) -5: evaluating the prediction result;

step 2) -6: completing the construction of a software defect prediction model;

further, the specific steps of the step 3) are as follows:

step 3) -1: an initial state;

step 3) -2: selecting a Go model as a defect discovery model;

step 3) -3: carrying out parameter expansion on the Go model;

step 3) -4: completing the construction of a software defect discovery model;

further, the specific steps of the step 4) are as follows:

step 4) -1: an initial state;

step 4) -2: setting residual defect rate R₁And R₂A value of (d);

step 4) -3: to V₀Each module m in the version_iEstimating a parameter a_iA value of (d);

step 4) -4: estimate V₀Version b₀A value;

step 4) -5: software defect discovery model V₀Finishing the estimation of the version parameters;

further, the specific steps of the step 5) are as follows:

step 5) -1: an initial state;

step 5) -2: reading V₀Each module m of the version_iA of (a)_iValue and defect prediction value p_i；

Step 5) -3: judgment V₁Each module M of a version_iAt V₀Whether the version exists or not, if so, executing the step 5) -4, otherwise, executing the step 5) -5;

step 5) -4: reading V₁Version of module M_iTo V₀Version of module m_iK value and m of defect repair information_iParameter a of_iValue and estimate V accordingly₁Version of module M_iParameter a of_iA value of (d);

step 5) -5: reading V₁Version of module M_iDefect prediction value p of_iAnd estimate V accordingly₁Version of module M_iParameter a of_iA value of (d);

step 5) -6: reading V₀Version b₀Value V₁Each module M of a version_iS of_iValue and estimate V accordingly₁Version of module M_iParameter b of_iA value of (d);

step 5) -7: software defect discovery model V₁Finishing the estimation of the version parameters;

further, the specific steps of the step 6) are as follows:

step 6) -1: an initial state;

step 6) -2: reading V₁Each module m of the version_iA of (a)_iValue b and_ia value;

step 6) -3: calculating V₁Each module m of the version_iA of (a)_iValue and all modules are according to A_iSorting the values from big to small;

step 6) -4: let k equal to 1;

step 6) -5: calculating λ according to equation 6.3_kA value of (d);

step 6) -6: if A is_k≥λ_k＞＝A_k+1Then λ^*＝λ_kExecuting steps 6) -7; otherwise, enabling k to be k +1, and returning to execute the steps 6) -5;

step 6) -7: to V₁Each module m of the version_iComputingA value;

step 6) -8: to V₁Each module m of the version_iIs pressed againstValue-assigning a test workload;

step 6) -9: software V₁The workload of the version optimal test is distributed;

the invention relates to a test workload distribution method based on a software reliability growth model, which uses methods such as random forest, linear regression, support vector machine and the like to establish a prediction model, uses a Go model as a defect discovery model, and uses V in the parameter estimation process of a cross-version defect discovery model₀Go model parameter of version to V₁Estimating the parameters of the version, applying the estimated parameters to a defect discovery model to finally obtain V₁The method further improves the software testing work efficiency, so that the most optimal testing workload distribution scheme of the version can be found under the condition of limited testing resourcesAnd the defects are more, so that the quality of the product is better controlled.

Drawings

Fig. 1 is a flowchart of a test workload distribution method based on a software reliability growth model according to an embodiment of the present invention.

FIG. 2 is a flow chart of the collection and processing of the software data set of FIG. 1.

FIG. 3 is a flow chart of the construction of the software defect prediction model of FIG. 1.

Fig. 4 is a flowchart of the construction of the software defect discovery model in fig. 1.

FIG. 5 is a model V for finding software defects in FIG. 1₀A flow chart of version parameter estimation.

FIG. 6 is a model V for finding software defects in FIG. 1₁A flow chart of version parameter estimation.

FIG. 7 shows the software V in FIG. 1_iA flow chart for version-optimal test workload distribution.

Detailed Description

In order to better understand the technical content of the present invention, specific embodiments are described below with reference to the accompanying drawings.

A test workload distribution method based on a software reliability growth model comprises the following steps:

s101 acquisition and processing of a software data set in order to be able to adapt the current system V₁To perform the test workload distribution, we need to do the previous version V₀The information of (2) is collected, specifically comprises defect information collection and test casesProcess statistics and metric collection; also for V₁The information of (2) is collected, specifically including measurement collection, setting of available testing workload and the like;

s102, constructing a software defect prediction model, namely after the step 1), converting V into V₀And V₁The measurement information of each module is organized into a form of a feature vector, the defect number is used as a target variable, and V is subjected to prediction model pair through random forest, linear regression, support vector machine and the like₀Set up training and for V₁Carrying out prediction;

s103, constructing a software defect finding model which can estimate the number of the detectable defects relative to the given test resource, wherein the model used is a Go model, because the model is the simplest NHPP model and has a constant defect detection rate in any test time, which means that the parameter estimation of the model is easier than that of other SRGM;

s104 software defect discovery model V₀Version parameter estimation, we need to estimate a from the existing data of acceptance test_iAnd b₀；

S105 software defect finding model V₁Version parameter estimation due to V₁The actual defect number of the version is unknown, so the method in step 4) cannot be used to estimate V₁Parameters in the version, we adopt the method of using V₀To estimate V₁The parameters of (1);

s106 software V₁Version-optimal test workload distribution, for Go model, module m is known_iA of (a)_iAnd b_iAnd V_iTest workload t to be allocated for a version_totalThen the optimal test workload distribution strategy can be obtained and distributed to m according to the strategy_iSubstituting the corresponding workload into a defect finding model to finally find the most defects;

FIG. 2 is a flow chart of the collection and processing of the software data set of FIG. 1 in order to enable reconciliationFront system V₁To perform the test workload distribution, we need to do the previous version V₀Collecting the information, specifically including defect information collection, test case processing and measurement collection; also for V₁The information of (2) is collected, and specifically comprises measurement collection, setting of available testing workload and the like. The method comprises the following specific steps:

step 1: an initial state; step 2: to V₀Collecting the defect information, and counting the actual defect number of each module; and step 3: to V₀To ensure V₀Can run for V₀Manually ensuring that there is a test case that can trigger the defect; specifically, if a test case which can trigger a certain defect cannot be found for the defect, the defect is excluded from statistics; and 4, step 4: statistics V₀Code metric and process metric information for each module of (a); and 5: statistics V₁Code metric and process metric information for each module of (1), initialization V₁A value of available test workload; step 6: the collection and processing of the software data set are finished;

FIG. 3 is a flow chart of the construction of the software defect prediction model in FIG. 1, after step 1), V is set₀And V₁The measurement information of each module is organized into a form of a feature vector, the defect number is used as a target variable, and V is subjected to prediction model pair through random forest, linear regression, support vector machine and the like₀Set up training and for V₁And (6) performing prediction. The method comprises the following specific steps:

step 1: an initial state; step 2: will V₀And V₁The measurement information of each module is organized into a form of a feature vector, and the defect number is used as a target variable; and step 3: with V₀Establishing a prediction model for the training set; and 4, step 4: with V₁For test set, pair V₁Each module in the set outputs a defect prediction value; and 5: evaluating the prediction result; step 6: completing the construction of a software defect prediction model;

fig. 4 is a flow chart of the construction of the software defect discovery model of fig. 1, which can estimate the number of discoverable defects with respect to a given test resource, and the model we use is the Go model, which is the simplest NHPP model, with a constant defect detection rate at any test time, which means that the parameter estimation of the model is much easier than other SRGMs. The method comprises the following specific steps:

step 1: an initial state; step 2: selecting a Go model as a defect discovery model; and step 3: expanding a Go model; and 4, step 4: and finishing the construction of the software defect finding model.

FIG. 5 is a model V for finding software defects in FIG. 1₀A flow chart of version parameter estimation. Due to V₁The actual defect number of the version is unknown, so the method in step 4) cannot be used to estimate V₁Parameters in the version, we adopt the method of using V₀To estimate V₁The parameter (c) of (c). The method comprises the following specific steps:

step 1: an initial state; step 2: setting residual defect rate R₁And R₂A value of (d); and step 3: to V₀Each module m in the version_iEstimating a parameter a_iA value of (d); and 4, step 4: estimate V₀Version b₀A value; and 5: software defect discovery model V₀And finishing the estimation of the version parameters.

FIG. 6 is a model V for finding software defects in FIG. 1₁A flow chart of version parameter estimation. Due to V₁The actual defect number of the version is unknown, so the method in step 4) cannot be used to estimate V₁Parameters in the version, we adopt the method of using V₀To estimate V₁The parameter (c) of (c). The method comprises the following specific steps:

step 1: an initial state; step 2: reading V₀Each module m of the version_iA of (a)_iValue and defect prediction value p_i(ii) a And step 3: judgment V₁Each module M of a version_iAt V₀Whether the version exists or not, if so, executing the step 4, otherwise, executing the step 5; and 4, step 4: reading V₁Version of module M_iTo V₀Version of module m_iK value and m of defect repair information_iParameter a of_iValue and estimate V accordingly₁Version of module M_iParameter a of_iA value of (d); and 5: reading V₁Version of module M_iDefect prediction value p of_iAnd estimate V accordingly₁Version of module M_iParameter a of_iA value of (d); step 6: reading V₀Version b₀Value V₁Each module M of a version_iS of_iValue and estimate V accordingly₁Version of module M_iParameter b of_iA value of (d); and 7: software defect discovery model V₁And finishing the estimation of the version parameters.

FIG. 7 shows the software V in FIG. 1₁A flow chart for version-optimal test workload distribution. For the Go model, module m is known_iA of (a)_iAnd b_iAnd V₁Test workload t to be allocated for a version_totalThen the optimal test workload distribution strategy can be obtained and distributed to m according to the strategy_iAnd substituting the corresponding workload into the defect finding model to finally find the most defects. The method comprises the following specific steps:

step 1: an initial state; step 2: reading V₁Each module m of the version_iA of (a)_iValue b and_ia value; and step 3: calculating V₁Each module m of the version_iA of (A)_iValue and all modules are according to A_iSorting the values from big to small; and 4, step 4: calculating lambda^*The value of (c). And 5: to V₁Each module m of the version_iComputingStep 6: to V₁Each module m of the version_iIs pressed againstValue-assigning a test workload; and 7: software V₁And finishing the distribution of the workload of the optimal version test.

In summary, the invention solves the problem of the defects existing in the maximum benefit testing system under the condition of limited testing resources at present, firstly, the defects of various defect prediction models are adopted for prediction, then, the cross-version parameter estimation is carried out by means of the prediction result, and finally, the optimal testing workload distribution result set of the defect discovery models under the parameter condition is solved.

Claims

1. A test workload distribution method based on software reliability growth model is characterized in that the old version (V) of software is reported through defect patches₀Version) and new version (V)₁Version) to respectively perform the collection and processing processes of the data set; then, a defect prediction model (V) is constructed by using defect prediction technologies such as random forest, linear regression and support vector machine₀For the training set, V₁As a test set); then use Goel-Okumoto model, one of the software reliability growth models, to pair V₀Version and V₁Version independent build software deficiencyTrapping a discovery model, and respectively carrying out parameter estimation on the defect discovery models of the two versions; finally, software V can be obtained₁An optimal test workload (test case) allocation scheme of the version; the method comprises the following steps:

1) collecting and processing a software data set; software version V₀The data information to be collected includes: collecting defect information, collecting and processing test cases and collecting basic measurement; software version V₁The data information to be collected includes: collecting basic measurement, distributing total testing workload and the like;

2) constructing a software defect prediction model;

definition 1: the software defect prediction model comprises the following three key attributes, namely description, input and output;

the following steps are described: the defect prediction model is used for reading a software training data set for training and testing a software testing set, and common prediction models comprise random forests, linear regression, support vector machines and the like;

inputting: code measurement, process measurement and other information of the software;

and (3) outputting: indicating the number of defects expected to occur in the software;

using the software data set collected and processed in step 1), respectively converting the version V₀And version V₁As a training set and a testing set, and organizing according to the format of description, input and output to carry out the software defect number prediction process;

3) constructing a software defect discovery model;

definition 1: the software defect discovery model can estimate the number of the discoverable defects relative to a given test resource, the model used by the software defect discovery model is an expanded exponential reliability growth model, also called a Goel-Okumoto model (hereinafter referred to as a Go model), the model has a constant defect detection rate in any test time, the model can represent the relationship between the test workload and the detected accumulated defects, and the calculation formula is as follows:

in the formulaFor the module m given a certain amount of test workload_iNumber of defects, t, that can be found finally_iIs a module m_iAssigned test workload, b_iNumber of defects that can be found for a unit test workload, a_iIs a module m_iInitial defect number of S_iIs a module m_iSize of (a) b₀Is constant and is estimated by step 4).

4) Software defect discovery model V₀Estimating version parameters;

definition 1: software defect discovery model V₀Version a_iThe calculation formula is as follows:

wherein H_iIs a module m_iThe number of actually detected defects of the optical disc,is a module m_iThe number of added/modified code lines,is a module m_iThe number of code lines multiplexed in (1). R₁And R₂Is the residual defect rate, which indicates that R still remains in the 1KLOC line in the newly added/modified code₁A defect; in the multiplexed code, R still remains in the 1KLOC line₂And (4) a defect.

Definition 2: parameter b₀The number of defects detectable per unit of test workload is indicated, which is derived from the actually assigned test workload and the number of detected defects, and the calculation formula is as follows:

b₀＝-S_total/t_total*log(1-H_total/a_total)

5) Software defect discovery model V₁Estimating version parameters;

case 1: if m_iAt V₀In existence of

A. If it is notSuppose V₀Version m_iHas k defects at V₁Is repaired, then:

B. if it is notDescription of m_iPractically without defects, so we can lower V₁Version m_iA of (a)_iThe value:

a_i(v_i，m_i)＝a_i(v_i-1，m_i)/10

wherein model is at V₀The defect prediction model, Aplymodel (model, m), established above_i) M predicted for applying the defect prediction model_iThe number of defects above, instead of the number of defects actually detected.

6) Software V₁Distributing the workload of the version optimal test;

known as V₁Test workload t to be allocated for a version_totalEach module m_iA of (a)_iAnd b_iThe value m under the optimal test workload distribution strategy can be obtained according to the steps_iThe results are substituted into the defect discovery model and the maximum number of accumulated defects can be finally discovered.

2. Software-based according to claim 1The method for distributing the testing workload of the reliability growth model is characterized in that in the step 1), the version V is subjected to₀The test cases are processed, and V is counted₀Actual test workload of; and for V₀Manually ensuring that there is a test case that can trigger the defect; specifically, if a test case which can trigger a certain defect cannot be found for the defect, the defect is excluded from statistics; then statistics version V₀And V₁Code metrics and process metrics information for each module of (a).

3. The software reliability growth model-based test workload distribution method according to claim 1, wherein in step 2), the processed software data set is subjected to a test; respectively convert version V₀And version V₁And the software defect number prediction process is performed by being used as a training set and a test set and being organized according to the format of description, input and output.

4. The method as claimed in claim 1, wherein in step 3), the Go model is selected as the defect discovery model and the Go model is subjected to parameter expansion, because the Go model is the simplest NHPP model and has a constant defect detection rate at any testing time.

5. The software reliability growth model-based test workload distribution method according to claim 1, wherein in step 4), for V₀Version, residual defect rate R of given software₁And R₂Can be in accordance with V₀The existing defect information and basic metric information of each module are versioned to estimate a_iAnd b₀The value of (c).

6. The method of claim 1, wherein the method comprises the step of distributing the test workload based on the software reliability growth modelIn step 5), due to V₁The actual defect number of the version is unknown, so the method in step 4) cannot be used to estimate V₁Parameters in the version, we adopt the method of using V₀To estimate V₁The parameters of (1); the specific steps are that V is read first₀Each module m of the version_iA of (a)_iValue and defect prediction value p_i(ii) a Then, judging V₁Each module M of a version_iAt V₀Whether or not there are: if so, read M_iTo V₀Corresponding module m in version_iDefect repair information (k value) and a_iValue to estimate M_iParameter a of_iA value of (d); otherwise read M_iDefect prediction value p of_iValue to estimate M_iParameter a of_iA value of (d); next, estimate M_iB of (a)_iValue by reading V₀Version b₀Value (as V)₁Version b₀Value) and M_iS of_iValue and estimate M accordingly_iParameter b of_iThe value of (c).

7. The software reliability growth model-based test workload distribution method according to claim 1, wherein in step 6), V is known for Go model₁Test workload t to be allocated for a version_totalModule M_iA of (a)_iAnd b_iThen, the optimal distribution strategy of the test workload to M can be obtained_iThe workload of (2) is substituted into the defect discovery model, and finally the maximum number of accumulated defects can be discovered.