CN110825620A - Test case priority calculation method in regression test - Google Patents
Test case priority calculation method in regression test Download PDFInfo
- Publication number
- CN110825620A CN110825620A CN201910980833.1A CN201910980833A CN110825620A CN 110825620 A CN110825620 A CN 110825620A CN 201910980833 A CN201910980833 A CN 201910980833A CN 110825620 A CN110825620 A CN 110825620A
- Authority
- CN
- China
- Prior art keywords
- module
- calculating
- risk
- value
- modules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
Abstract
The invention provides a method for calculating the priority of a test case in a regression test, which calculates a module failure possibility value according to the change times of requirements and the like; calculating a module failure impact value according to the calling relation and the importance degree of the module; and finally, obtaining the risk value of the test case, and sequencing the risk value to obtain the priority of the test case. The calculation basis in the method reflects the concerned problem in the regression test to a great extent, and the module failure possibility value and the module failure impact value can reflect the influence caused by code abnormality and are more consistent with the actual situation; the method has the characteristics of simplicity, high efficiency and the like, and is suitable for unit test and integrated test.
Description
Technical Field
The invention belongs to the field of software testing in software engineering, and particularly relates to a method for calculating the priority of a test case in regression testing.
Background
In software engineering practice, a software testing link is crucial; regression testing is a very important step in software testing, namely testing software after software code is modified; the most reliable regression testing method is to execute regression testing on all test cases, but under the conditions of time tightness, budget limitation and the like, the execution of regression testing on all test cases is unrealistic, so that how to selectively execute regression testing is an important content in software testing.
The test case selection method based on the priority is a common method in regression test case selection, can calculate the priority according to factors such as past test results, coverage rate of each module, historical failure rate and the like, can also calculate the priority based on module risks, and has various effects in practical application.
The test case priority calculation method based on module risk measures a module risk value by calculating the possibility of error occurrence and the harm brought by the error, thereby determining the priority; module risk can be generally expressed as two indicators, one being the probability of module failure or the likelihood of failure, and the other being the loss or impact of module failure; the evaluation of the two indexes depends on experience in many cases, and manual processing is needed, which greatly affects the efficiency of the test, and meanwhile, more manual processing also brings uncertainty of the effect.
In order to solve the problems, the invention provides a method for calculating the priority of the test case in the regression test, which can calculate the risk factor of the module and the risk value of the test case before the regression test according to the factors such as user requirements, requirement change, codes, modification conditions and the like, thereby realizing the calculation of the priority of the regression test case and having better application effect.
Disclosure of Invention
The method for calculating the priority of the test case in the regression test comprises the following steps.
Step 1, collecting and counting code change conditions caused by demand change by taking a module as a unit;
the module may be a function or a method for implementing a specific function, or may be a class or a set including a plurality of specific functions or methods.
Step 2, calculating risk factors caused by requirement change of the module;
the change of the requirement is an important factor causing code modification, and the modification times and the modification degree of the code in the module are important factors influencing the risk of the module; therefore, the more changes in the requirements a module covers, the higher the risk level;
the risk factor of a module due to a change in demand may be represented by the ratio of the number of changes in demand for the module to the total number of changes in demand.
Step 3, calculating risk factors caused by complexity of the module;
in general, a complex module usually contains more requirements, and in turn, a module containing more requirements is more complex and has more risks;
the risk factor of a module due to the complexity of the demand can be represented by the ratio of the demand term contained in the module to the total demand term.
Step 4, calculating risk factors of the module caused by code complexity;
in general, the number of judgments and loops used in a code affects the complexity of a module, and the more complex the logical relationship, the greater the risk thereof, which can be measured by the degree of complexity of the circles;
the risk factor of a module due to code complexity can be expressed as the ratio of the module's round-robin to the maximum round-robin for all modules.
Step 5, calculating risk factors of the module caused by the code amount;
generally, the more the number of lines of code, the greater the potential risk; therefore, the code line number is also an important index for measuring the risk of the module;
the risk factor of a module due to the code amount can be represented by the ratio of the number of code lines of the module to the maximum number of code lines in all modules.
Step 6, calculating a module failure possibility value;
the module failure means that the module has operation errors in the operation process and does not reach the behavior of the expected function;
the module failure probability value can be represented by the product of the risk factors of the modules calculated in the steps 2-5.
Step 7, counting all called modules of the module;
the calling relation among the modules reflects the dependency relation among the modules in the software; if a module contains errors, it is obvious that a module which is directly or indirectly dependent on the module will also be affected in the event of a fault;
all called modules of a module are represented as a set of all called modules in the module, namely a sub-module set; a static set of sub-modules can be obtained by simply scanning the code.
Step 8, manually appointing an importance value of the module;
the importance of a module characterizes how much the module should be valued, and may be specified manually.
Step 9, calculating a module failure impact value;
after a module fails, all other modules calling the module are affected;
the module failure impact value may be expressed as a ratio of the average of the importance of each module in its set of sub-modules to the maximum of the importance of the entire module.
Step 10, calculating a risk factor of a module;
the risk factor for a module may be expressed as a product of a module failure likelihood value and a module failure impact value.
Step 11, calculating the risk value and the priority of the test case;
the unit test case may only comprise one module when tested;
the integrated test case may comprise a plurality of modules when tested;
calculating and accumulating risk factors of all modules contained in the test case to be used as a risk value of the test case;
sequencing the test cases according to the risk values, wherein the priority with the maximum risk value is highest;
therefore, the calculation of the priority of the test cases and the sequencing of the test cases are realized.
Aiming at the priority problem of a test case in a regression test, firstly, calculating a module failure possibility value according to the change times of requirements, the complexity of the requirements, the complexity of codes and the code quantity; calculating a module failure impact value according to the calling relation and the importance degree of the module; taking the product of the two as a risk factor of the module; and accumulating the risk factors of all modules contained in the test cases to obtain the risk values of the test cases, and sequencing the risk values to obtain the priority of the test cases. The calculation basis in the method reflects the concerned problem in the regression test to a great extent, and the module failure possibility value and the module failure impact value can reflect the influence caused by code abnormality and are more consistent with the actual situation; the method has the characteristics of simplicity, high efficiency and the like, is suitable for unit test and integrated test, and has a good application effect.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
Detailed Description
In order to more clearly show the objects and technical solutions of the present invention, the following will describe the specific embodiments and parameters of the present invention in more detail with reference to the specific implementation processes and the attached drawings.
The method for calculating the priority of the test case in the regression test, as shown in fig. 1, includes the following steps.
Step 1, collecting and counting code change conditions caused by demand change by taking a module as a unit;
the module may be a function or a method for implementing a specific function, or may be a class or a set including a plurality of specific functions or methods.
Step 2, calculating risk factors caused by requirement change of the module;
the risk factor of a module due to a change in demand may be represented by the ratio of the number of changes in demand for the module to the total number of changes in demand.
And if the number of the requirement changes related to a certain module is RCi and the total number of the requirement changes in the regression test is NRC, the risk factor of the module caused by the requirement changes can be represented as RCi/NRC, and the value range is 0-1.
If the number of changes of the demand of a certain module is 3 and the total number of changes of the demand of the current regression is 100, the risk factor is 0.03.
Step 3, calculating risk factors caused by complexity of the module;
the risk factor of a module due to the complexity of the demand can be represented by the ratio of the demand term contained in the module to the total demand term.
Setting a requirement item contained in a certain module as Ri and setting a total requirement item as NRV; the risk factor of the module due to the complexity of the demand can be expressed as Ri/NRV, which is in the range of 0 to 1.
If a module requirement item is 5 and the total requirement item is 100, the risk factor is 0.05.
Step 4, calculating risk factors of the module caused by code complexity;
the risk factor of a module due to code complexity can be expressed as the ratio of the module's round-robin to the maximum round-robin for all modules.
And if the circle complexity of a certain module is Ci and the maximum circle complexity of all modules is MCC, the risk factor of the module caused by code complexity can be represented as Ci/MCC, and the value range is 0-1.
If the module complexity is 10 and the maximum complexity is 100, the risk factor is 0.10.
Step 5, calculating risk factors of the module caused by the code amount;
the risk factor of a module due to the code amount can be represented by the ratio of the number of code lines of the module to the maximum number of code lines in all modules.
And if the code line number of a certain module is Li and the maximum value of the line numbers in all the modules is MLC, the risk factor of the module caused by the code amount can be represented as Li/MLC, and the value range is also 0-1.
If the code line number of a module is 50 and the maximum code behavior is 200, the risk factor is 0.40.
Step 6, calculating a module failure probability value, which can be expressed as the product of the factors calculated in steps 2 to 5, namely: RCi/NRC × Ri/NRV × Ci/MCC × Li/MLC.
According to the data of the steps 2-5, the risk factor of a certain module caused by the requirement change is 0.03; the risk factor due to the complexity of the requirements is 0.05; the risk factor due to code complexity is 0.10; the risk factor due to the amount of code is 0.40; the module failure probability value is: 0.03 × 0.05 × 0.1 × 0.4=6 e-5.
Step 7, counting all called modules of the module;
all called modules of a module are represented as a set of all called modules in the module, namely a sub-module set;
the sub-module is the called module; the static submodule set can be obtained by simply scanning the codes;
scanning a source code, and if a certain module comprises a called module, adding the called module into the submodule set;
all called sub-modules of the sub-modules also need to be added, so that the recursion and circulation during scanning are noticed, and the sub-modules are not repeated during statistics;
if the M1 module calls M3 and M5, and M5 calls M6 and M7, the set is: { M3, M5, M6, M7 }.
Step 8, manually appointing an importance value of the module;
the importance of the module characterizes the degree to which the module should be valued, and is manually specified;
the importance value range of the module can be set to 1-5, with 5 being the most important and 1 being the least important.
Step 9, calculating a module failure impact value;
the module failure impact value can be expressed as the ratio of the average value of the importance of each module in the sub-module set to the maximum value of the importance of all modules;
and if the average value of the importance of each module in the sub-module set of a certain module is Ii and the maximum value MIV, the impact value caused by the failure of the module can be represented as Ii/MIV, and the range of the impact value is 0-1.
According to the example in step 7, assuming that the importance of the modules M3, M5, M6 and M7 are 2, 3, 1 and 4, respectively, the average value of the importance in the sub-module set of M1 is: (2+3+1+4)/4= 2.5; assuming a maximum of 4 for the importance of all modules, the failure impact value is 2.5/4= 0.625.
Step 10, calculating a risk factor of a module;
the risk factor for a module may be expressed as a product of the probability of module failure value and the impact value due to module failure.
If the module failure probability value is 6e-5 and the impact value caused by failure is 0.625 according to the data of the steps 2 to 9, the risk factor of the M1 module is as follows: 3.75 e-5.
Step 11, calculating the risk value and the priority of the test case;
the unit test case may only comprise one module when tested;
the integrated test case may comprise a plurality of modules when tested;
calculating and accumulating risk factors of all modules contained in the test case to be used as a risk value of the test case;
sequencing the test cases according to the risk values, wherein the priority with the maximum risk value is highest;
therefore, the calculation of the priority of the test cases and the sequencing of the test cases are realized.
For example, if test case T1 includes modules M1 and M2, test case T2 includes modules M3 and M4, and risk factors of modules M1, M2, M3 and M4 are 3.75e-5, 1e-5, 3e-5 and 2e-6, respectively, then the risk value of the test case T1 is 4.75e-5, the risk value of the test case T2 is 3.2e-5, the risk value of the test case T1 is greater than that of the test case T2, and the priority is greater than that of the test case T2.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the foregoing embodiments illustrate the present invention in detail, those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; or modifying the use order of various steps; the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention; the values of the various thresholds and ranges of the present invention may vary depending on the particular situation.
Claims (10)
1. The method for calculating the priority of the test case in the regression test is characterized by comprising the following steps of:
step 1, collecting and counting code change conditions caused by demand change by taking a module as a unit;
step 2, calculating risk factors caused by requirement change of the module;
step 3, calculating risk factors caused by complexity of the module;
step 4, calculating risk factors of the module caused by code complexity;
step 5, calculating risk factors of the module caused by the code amount;
step 6, calculating a module failure possibility value;
step 7, counting all called modules of the module;
step 8, manually appointing an importance value of the module;
step 9, calculating a module failure impact value;
step 10, calculating a risk factor of a module;
step 11, calculating the risk value and the priority of the test case;
therefore, the calculation of the priority of the test cases and the sequencing of the test cases are realized.
2. In step 2, the calculating module calculates a risk factor caused by the change of the demand, including:
the risk factor of a module due to a change in demand may be represented by the ratio of the number of changes in demand for the module to the total number of changes in demand.
3. Step 3, calculating risk factors caused by the complexity of the module, including:
the risk factor of a module due to the complexity of the demand can be represented by the ratio of the demand term contained in the module to the total demand term.
4. Step 4, calculating the risk factor of the module caused by the code complexity, including:
the risk factor of a module due to code complexity can be expressed as the ratio of the module's round-robin to the maximum round-robin for all modules.
5. In step 5, calculating a risk factor of the module due to the code amount, including:
the risk factor of a module due to the code amount can be represented by the ratio of the number of code lines of the module to the maximum number of code lines in all modules.
6. In step 6, calculating a module failure probability value, including:
the module failure probability value may be expressed as a product of the respective module risk factors calculated in steps 2 through 5.
7. In step 7, all the called modules of the statistical module include:
all called modules of a module are represented as a set of all called modules in the module, namely a sub-module set; a static set of sub-modules can be obtained by simply scanning the code.
8. Step 9, calculating a module failure impact value, including:
the module failure impact value may be expressed as a ratio of the average of the importance of each module in its set of sub-modules to the maximum of the importance of the entire module.
9. In step 10, calculating a risk factor of the module, including:
the risk factor for a module may be expressed as a product of a module failure likelihood value and a module failure impact value.
10. In step 11, calculating the risk value and priority of the test case includes:
calculating and accumulating risk factors of all modules contained in the test case to be used as a risk value of the test case;
and sequencing the test cases according to the risk values, wherein the priority with the maximum risk value is highest.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910980833.1A CN110825620B (en) | 2019-10-16 | 2019-10-16 | Test case priority calculation method in regression test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910980833.1A CN110825620B (en) | 2019-10-16 | 2019-10-16 | Test case priority calculation method in regression test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110825620A true CN110825620A (en) | 2020-02-21 |
| CN110825620B CN110825620B (en) | 2021-06-15 |
Family
ID=69549446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910980833.1A Active CN110825620B (en) | 2019-10-16 | 2019-10-16 | Test case priority calculation method in regression test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110825620B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590472A (en) * | 2021-07-06 | 2021-11-02 | 四川大学 | Test case priority ranking method in regression test |
| CN113760769A (en) * | 2021-09-13 | 2021-12-07 | 北京百度网讯科技有限公司 | Test case processing method and device, electronic equipment and storage medium |
| CN113886252A (en) * | 2021-09-30 | 2022-01-04 | 四川大学 | Regression test case priority determination method based on thermodynamic diagram |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833501A (en) * | 2010-04-02 | 2010-09-15 | 中国科学院软件研究所 | Newly increased requirement-based code variation quantitative evaluation method and system thereof |
| CN102880545A (en) * | 2012-08-30 | 2013-01-16 | 中国人民解放军63928部队 | Method for dynamically adjusting priority sequence of test cases |
| CN103298016A (en) * | 2012-02-27 | 2013-09-11 | 展讯通信（上海）有限公司 | Test system for mobile terminals |
| CN104123227A (en) * | 2014-08-13 | 2014-10-29 | 广东电网公司信息中心 | Method for automatically generating testing cases |
| US20150347278A1 (en) * | 2014-05-28 | 2015-12-03 | Vmware, Inc. | Identifying test gaps using code execution paths |
| CN105302710A (en) * | 2014-07-03 | 2016-02-03 | 腾讯科技（深圳）有限公司 | Method and apparatus for determining test case in need of regression testing |
| CN105446885A (en) * | 2015-12-28 | 2016-03-30 | 西南大学 | Regression testing case priority ranking technology based on needs |
| US9424167B2 (en) * | 2014-05-21 | 2016-08-23 | Cgi Technologies And Solutions Inc. | Automated testing of an application system |
| US10229038B2 (en) * | 2016-03-15 | 2019-03-12 | International Business Machines Corporation | Generating reusable testing procedures |
| CN109947649A (en) * | 2019-03-20 | 2019-06-28 | 北京邮电大学 | A kind of sort method and device of regression test case |
| CN110109820A (en) * | 2019-03-19 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Regression test case determines method, apparatus, computer equipment and storage medium |
-
2019
- 2019-10-16 CN CN201910980833.1A patent/CN110825620B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833501A (en) * | 2010-04-02 | 2010-09-15 | 中国科学院软件研究所 | Newly increased requirement-based code variation quantitative evaluation method and system thereof |
| CN103298016A (en) * | 2012-02-27 | 2013-09-11 | 展讯通信（上海）有限公司 | Test system for mobile terminals |
| CN102880545A (en) * | 2012-08-30 | 2013-01-16 | 中国人民解放军63928部队 | Method for dynamically adjusting priority sequence of test cases |
| US9424167B2 (en) * | 2014-05-21 | 2016-08-23 | Cgi Technologies And Solutions Inc. | Automated testing of an application system |
| US20150347278A1 (en) * | 2014-05-28 | 2015-12-03 | Vmware, Inc. | Identifying test gaps using code execution paths |
| US20170075795A1 (en) * | 2014-05-28 | 2017-03-16 | Vmware, Inc. | Identifying test gaps using code execution paths |
| CN105302710A (en) * | 2014-07-03 | 2016-02-03 | 腾讯科技（深圳）有限公司 | Method and apparatus for determining test case in need of regression testing |
| CN104123227A (en) * | 2014-08-13 | 2014-10-29 | 广东电网公司信息中心 | Method for automatically generating testing cases |
| CN105446885A (en) * | 2015-12-28 | 2016-03-30 | 西南大学 | Regression testing case priority ranking technology based on needs |
| US10229038B2 (en) * | 2016-03-15 | 2019-03-12 | International Business Machines Corporation | Generating reusable testing procedures |
| CN110109820A (en) * | 2019-03-19 | 2019-08-09 | 深圳壹账通智能科技有限公司 | Regression test case determines method, apparatus, computer equipment and storage medium |
| CN109947649A (en) * | 2019-03-20 | 2019-06-28 | 北京邮电大学 | A kind of sort method and device of regression test case |
Non-Patent Citations (2)
| Title |
|---|
| FENG ZILIANG: ""Version specific test case prioritization"", 《JOURNAL OF INTELLIGENT & FUZZY SYSTEMS》 * |
| 陈翔等: ""回归测试中的测试用例优先排序技术述评"", 《软件学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113590472A (en) * | 2021-07-06 | 2021-11-02 | 四川大学 | Test case priority ranking method in regression test |
| CN113760769A (en) * | 2021-09-13 | 2021-12-07 | 北京百度网讯科技有限公司 | Test case processing method and device, electronic equipment and storage medium |
| CN113760769B (en) * | 2021-09-13 | 2023-11-07 | 北京百度网讯科技有限公司 | Test case processing method and device, electronic equipment and storage medium |
| CN113886252A (en) * | 2021-09-30 | 2022-01-04 | 四川大学 | Regression test case priority determination method based on thermodynamic diagram |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110825620B (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110825620B (en) | Test case priority calculation method in regression test | |
| CN107066365B (en) | System abnormity monitoring method and device | |
| US20130086010A1 (en) | Systems and methods for data quality control and cleansing | |
| CN110825621B (en) | Regression test case priority calculation method based on dynamic risk | |
| CN108920336B (en) | Service abnormity prompting method and device based on time sequence | |
| CN105224460A (en) | A kind of method based on Program instrumentation statistical test coverage rate | |
| CN109375151B (en) | Monitoring channel scheduling method and device for electric energy meter metering error on-line monitoring technology | |
| CN112311611B (en) | Data anomaly monitoring method and device and electronic equipment | |
| CN111103851B (en) | System and method for anomaly characterization based on joint history and time series analysis | |
| CN110781027B (en) | Method, device and equipment for determining error reporting threshold of memory ECC (error correction code) | |
| CN110737587B (en) | Test method and device based on test case, storage medium and server | |
| CN114495497A (en) | Method and system for distinguishing and interpolating traffic abnormal data | |
| WO2017082782A1 (en) | Managing network alarms | |
| CN113590429A (en) | Server fault diagnosis method and device and electronic equipment | |
| JP2012208644A (en) | Risk management device | |
| CN114444827A (en) | Cluster performance evaluation method and device | |
| CN113835947A (en) | Method and system for determining abnormality reason based on abnormality identification result | |
| CN110765005A (en) | Software reliability evaluation method and device | |
| CN117009115A (en) | Fault positioning method and device, electronic equipment and computer storage medium | |
| CN114676422A (en) | Resource access abnormity detection method, device and equipment | |
| CN108269004B (en) | Product life analysis method and terminal equipment | |
| CN111367640B (en) | Data statistics period determining method and device, electronic equipment and storage medium | |
| CN111371594B (en) | Equipment abnormality alarm method and device and electronic equipment | |
| CN109921867A (en) | A kind of bit error rate advance decision method and judgment device | |
| CN112506803B (en) | Big data testing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |