CN113590472A - Test case priority ranking method in regression test - Google Patents
Test case priority ranking method in regression test Download PDFInfo
- Publication number
- CN113590472A CN113590472A CN202110762419.0A CN202110762419A CN113590472A CN 113590472 A CN113590472 A CN 113590472A CN 202110762419 A CN202110762419 A CN 202110762419A CN 113590472 A CN113590472 A CN 113590472A
- Authority
- CN
- China
- Prior art keywords
- test cases
- test
- sequencing
- similarity
- test case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a test case priority ranking method in regression testing. Firstly, clustering test cases according to the text theme and coverage similarity of the test cases; then, taking the maximized code coverage rate, the historical execution failure rate and the minimized execution time as targets, and sequencing the test cases by using a multi-objective optimization algorithm and combining clustering results; and finally, dynamically adjusting the sequencing result by utilizing the incidence relation among the test cases to finally obtain the sequencing sequence of the priority of the test cases. The method integrates a data mining technology and a multi-objective optimization technology, combines static text information and dynamic execution information of the test cases, is a brand new scheme integrating the advantages of white-box test case priority ordering and black-box test case priority ordering, improves defect detection rate, is considered more comprehensively, and can obtain better ordering results through experimental verification.
Description
Technical Field
The invention belongs to the field of software testing in software engineering, and particularly relates to a test case priority ranking method in regression testing.
Background
Regression testing is a software test for verifying whether a program or code change adversely affects existing functions, and is very common in the software industry. However, test cases are typically large in size, and the cost of completely executing all test cases in a regression test can be high. The test cases are sorted according to the priority, so that the test cases which are more likely to detect faults can be preferentially executed, and greater fault detection capability is provided within available test time, thereby improving the test efficiency and reducing the time and resource overhead. How to effectively calculate the priority of the test case is an important content in the regression test.
The test case priority ranking method in the existing regression test usually determines the similarity of test cases based on single information, so that the similarity of the test cases in all aspects cannot be comprehensively considered; also, many sorting methods only consider ranking test cases that can find defects first, and do not consider whether they reveal different defects.
Disclosure of Invention
The invention has proposed the priority ranking method of test case in a regression test, calculate the similarity of the test case in conjuction with static and dynamic information of the test case at first, cluster the test case; then, combining the clustering result to adjust the multi-target optimization sequencing result; and finally, whether the test cases reveal the same defects is taken into consideration of the association rule, and the sequencing result of the test cases is further adjusted. Experiments prove that the method has a better sequencing effect.
In order to achieve the above purpose, the invention adopts a method for sequencing the priorities of test cases in a regression test, which comprises the following three steps:
step 1, clustering test cases according to the text theme similarity and the sentence coverage similarity of the test cases;
the step 1, clustering test cases according to the text theme similarity and the sentence coverage similarity of the test cases, comprises the following steps:
step 1.1, preprocessing a test case text, performing theme modeling on the preprocessed text, and calculating text theme similarity of the test case;
the reason for executing the step is that the test case text contains various information of the test case, so that the similarity of the test case in function can be judged from the viewpoint of static text information of the test case;
step 1.2, counting statement coverage conditions of the test cases, and calculating statement coverage similarity of the test cases;
the reason for executing the step is that the statement coverage condition describes the proportion and the degree of the test case testing of the program source code, so that the similarity of the program source code on the code coverage range can be judged from the angle of the dynamic execution information of the test case;
step 1.3, determining the weighting coefficients of the two similarities according to experiments, and calculating the weighted similarity sum;
step 1.4, carrying out hierarchical clustering on the test cases according to the weighted similarity sum to obtain a clustering result; dividing the test cases into N different classes according to the similarity;
step 2, performing multi-target sequencing on the test cases, and adjusting a sequencing sequence according to a clustering result;
the step 2 of performing multi-target sequencing on the test cases and adjusting the sequencing sequence according to the clustering result comprises the following steps:
step 2.1, performing multi-target sequencing on the test cases by using a multi-target genetic algorithm to obtain a sequencing sequence, wherein the sequencing targets are code coverage maximization, historical execution failure rate maximization and execution time maximization;
step 2.2, according to the clustering result obtained in the step 1, adjusting the sequencing sequence obtained in the step 2.1 to ensure that the test cases arranged in the front belong to different clustering categories;
the reason for performing this step is that the test cases for detecting the same error may be arranged at the front end of the sequence in the sorted sequence, so the sorting needs to be adjusted according to the clustering result to improve the error detection speed.
Step 3, mining association rules according to historical execution results of the test cases, and dynamically adjusting sequencing sequences;
the step 3 of mining association rules according to historical execution results of the test cases and dynamically adjusting the sequencing sequence comprises the following steps:
step 3.1, mining association rules of execution failure among the test cases according to the historical execution information of the test cases;
step 3.2, if the obtained association rule, the front piece and the back piece of the association rule reveal the same defect, discarding the association rule;
the reason for performing this step is that when one test case fails to execute, another test case has to be executed immediately only if the revealed defects are different.
3.3, executing the sequencing sequence obtained in the step 2, if the execution of a certain test case fails in the process, searching for an association rule taking the test case as a front piece, and adjusting the test case of a rear piece to be immediately executed;
the reason for executing the step is to ensure that after a certain test case fails, the following test case with execution failure association is immediately executed, and the test case reveals different defects so as to improve the error detection rate.
The resulting test case execution sequence is the prioritized sequence.
According to the text theme similarity and the sentence coverage similarity of the test cases, the method measures the similarity of the test cases from static and dynamic dimensions respectively and clusters the test cases; then, taking the maximized code coverage rate, the historical execution failure rate and the minimized execution time as targets, and sequencing the test cases by using a multi-objective optimization algorithm and combining clustering results; and finally, dynamically adjusting the sequencing result by utilizing the incidence relation among the test cases to finally obtain the sequencing sequence of the priority of the test cases. The method is a brand new scheme fusing a data mining technology and a multi-objective optimization technology, combines static information and dynamic execution information of test cases to carry out similarity judgment, improves defect detection rate, is more comprehensive in consideration, and can obtain better results.
Drawings
FIG. 1 is a schematic general flow diagram of the process of the present invention.
Detailed Description
In order to more clearly show the objects and technical solutions of the present invention, embodiments of the present invention will be described in more detail below with reference to the accompanying drawings.
A method for sorting the priority of test cases in regression testing, as shown in FIG. 1, generally includes the following three steps:
step 1, clustering test cases according to the text theme similarity and the sentence coverage similarity of the test cases;
the method comprises the steps of measuring the similarity of test cases respectively by using the similarity of text themes and the similarity of code coverage rates, combining the two similarities by using weights, clustering the test cases by using a hierarchical clustering method, and classifying the similar test cases into one class to obtain a clustering result.
The step 1, performing test case clustering according to the text topic similarity and the sentence coverage similarity of the test cases, as shown in fig. 1, includes the following steps:
step 1.1, preprocessing a test case text, performing theme modeling on the preprocessed text, and calculating text theme similarity of the test case;
the test case text preprocessing refers to extracting language data of the test case text and acquiring a service concern point in the test case;
the theme modeling means that a theme vector is used for representing the correlation between each test case and different semantic themes to approximate the functions of the test cases;
the text theme similarity of the test cases means that the text theme similarity between the test cases is measured by calculating the distance between the text theme vectors of the test cases, and the smaller the distance is, the more similar the test cases are represented.
Step 1.2, counting statement coverage conditions of the test cases, and calculating statement coverage similarity of the test cases;
the statement coverage condition of the test case refers to which statements in the source code of the program to be tested are executed when the test case is used for executing the program to be tested. The more the statement coverage of the two test cases coincide, the more similar they represent;
the statement coverage similarity of the test cases means that the similarity based on statement coverage between two test cases is calculated by using the Jacard distance, and the smaller the Jacard distance is, the more similar the test cases are represented.
Step 1.3, determining the weighting coefficients of the two similarities according to experiments, and calculating the weighted similarity sum;
if the text topic similarity St is assigned a weight of 0.7 and the sentence coverage similarity Sc is assigned a weight of 0.3, the weighted similarity sum is St × 0.7+ Sc × 0.3. The determination of the two weighting factors is obtained by experiments on the actual project data.
Step 1.4, carrying out hierarchical clustering on the test cases according to the weighted similarity sum to obtain a clustering result;
if the threshold value of the hierarchical clustering is set to be N, the test cases are divided into N different classes according to the similarity, and the N different classes are used as clustering results.
Step 2, performing multi-target sequencing on the test cases, and adjusting a sequencing sequence according to a clustering result;
the method comprises the steps of combining three targets of code coverage maximization, historical execution failure rate maximization and execution time minimization, using a multi-target genetic algorithm to carry out multi-target sequencing on test cases, and then adjusting a sequencing sequence according to a clustering result obtained in the step 1, so that the test cases at the front in the sequence all belong to different classes. And preliminarily obtaining a sequence result after the step is finished.
The step 2 of performing multi-target sequencing on the test cases and adjusting a sequencing sequence according to a clustering result, as shown in fig. 1, includes the following steps:
step 2.1, performing multi-target sequencing on the test cases by using a multi-target genetic algorithm to obtain a sequencing sequence, wherein the sequencing targets are code coverage maximization, historical execution failure rate maximization and execution time maximization;
the multi-target sequencing of the test cases refers to searching a compromise sequencing scheme in a plurality of sequencing targets so as to improve the defect detection rate of the test cases; and respectively calculating three target values by using the function expressions, and obtaining a test case sequencing sequence optimized according to the target values by using a multi-target genetic algorithm.
Step 2.2, according to the clustering result obtained in the step 1, adjusting the sequencing sequence obtained in the step 2.1 to ensure that the test cases arranged in the front belong to different clustering categories;
if N test cases of different categories exist in the clustering result, the sorting sequence is adjusted to that the first N test cases belong to different categories, and the N test cases are respectively sorted first in the category to which the test cases belong.
Step 3, mining association rules according to historical execution results of the test cases, and dynamically adjusting sequencing sequences;
the method comprises the steps of firstly mining the incidence relation of the test cases which fail at the same time from the historical execution information of the test cases by using an incidence rule mining technology, and then adjusting the sequencing result of the test cases obtained in the previous step according to the relation, so that the error uncovering rate of the test cases is improved.
The step 3, mining association rules according to historical execution results of the test cases, and dynamically adjusting the sequencing sequence, as shown in fig. 1, includes the following steps:
step 3.1, mining association rules of execution failure among the test cases according to the historical execution information of the test cases;
the association rule of execution failure means that every time a certain test case fails, some other test cases always fail. Such as the association rule of execution failure X → Y, means that when one test case X fails to execute, another test case Y also often fails at the same time.
Step 3.2, if the front piece and the back piece of the obtained certain association rule reveal the same defect, discarding the association rule;
if the association rule X → Y is obtained, the defects revealed by the front part X and the back part Y are the same, and then the defects are discarded.
3.3, executing the sequencing sequence obtained in the step 2, if the execution of a certain test case fails in the process, searching for an association rule taking the test case as a front piece, and adjusting the test case of a rear piece to be immediately executed;
if the execution of a certain test case X fails, searching an association rule X → Y taking X as a front part, and adjusting the test case Y of a rear part to be immediately executed;
the resulting test case execution sequence is the prioritized sequence.
Claims (2)
1. A method for sequencing the priority of test cases in regression test is characterized by comprising the following steps:
step 1, clustering test cases according to the text theme similarity and the sentence coverage similarity of the test cases;
step 2, performing multi-target sequencing on the test cases, and adjusting a sequencing sequence according to a clustering result;
step 3, mining association rules according to historical execution results of the test cases, and dynamically adjusting sequencing sequences;
therefore, the test case priority is sequenced.
2. The method according to claim 1, wherein the step 1 of clustering the test cases according to the text topic similarity and the sentence coverage similarity of the test cases comprises the following steps:
step 1.1, preprocessing a test case text, performing theme modeling on the preprocessed text, and calculating text theme similarity of the test case;
step 1.2, counting statement coverage conditions of the test cases, and calculating statement coverage similarity of the test cases;
step 1.3, determining the weighting coefficients of the two similarities according to experiments, and calculating the weighted similarity sum;
step 1.4, carrying out hierarchical clustering on the test cases according to the weighted similarity sum to obtain a clustering result;
the method according to claim 1, wherein the step 2 of performing multi-target sequencing on the test cases and adjusting the sequencing sequence according to the clustering result comprises the following steps:
step 2.1, performing multi-target sequencing on the test cases by using a multi-target genetic algorithm to obtain a sequencing sequence, wherein the sequencing targets are code coverage maximization, historical execution failure rate maximization and execution time maximization;
step 2.2, according to the clustering result obtained in the step 1, adjusting the sequencing sequence obtained in the step 2.1 to ensure that the test cases arranged in the front belong to different clustering categories;
the method according to claim 1, wherein the step 3 of mining association rules according to historical execution results of test cases and dynamically adjusting the sequencing sequence comprises the following steps:
step 3.1, mining association rules of execution failure among the test cases according to the historical execution information of the test cases;
step 3.2, if the front piece and the back piece of the obtained certain association rule reveal the same defect, discarding the association rule;
3.3, executing the sequencing sequence obtained in the step 2, if the execution of a certain test case fails in the process, searching for an association rule taking the test case as a front piece, and adjusting the test case of a rear piece to be immediately executed;
the resulting test case execution sequence is the prioritized sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110762419.0A CN113590472B (en) | 2021-07-06 | 2021-07-06 | Test case priority ranking method in regression test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110762419.0A CN113590472B (en) | 2021-07-06 | 2021-07-06 | Test case priority ranking method in regression test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113590472A true CN113590472A (en) | 2021-11-02 |
| CN113590472B CN113590472B (en) | 2023-03-14 |
Family
ID=78246287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110762419.0A Active CN113590472B (en) | 2021-07-06 | 2021-07-06 | Test case priority ranking method in regression test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113590472B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978974A (en) * | 2022-05-20 | 2022-08-30 | 国网重庆市电力公司电力科学研究院 | Wireless communication module testing method, device, equipment and storage medium |
| CN115526460A (en) * | 2022-09-09 | 2022-12-27 | 珠海安士佳电子有限公司 | Intelligent production test system for security monitoring camera |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011042282A1 (en) * | 2009-10-08 | 2011-04-14 | International Business Machines Corporation | Automated test execution plan generation |
| CN109783349A (en) * | 2018-12-10 | 2019-05-21 | 江苏大学 | A kind of priorities of test cases sort method and system based on dynamical feedback weight |
| CN110502447A (en) * | 2019-08-30 | 2019-11-26 | 西安邮电大学 | A kind of regression test case priority ordering method based on figure |
| CN110825620A (en) * | 2019-10-16 | 2020-02-21 | 四川大学 | Test case priority calculation method in regression test |
| CN111427802A (en) * | 2020-06-09 | 2020-07-17 | 南京大学 | Test method and system for carrying out test case priority sequencing by utilizing ensemble learning |
| CN112463636A (en) * | 2020-12-11 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Test case sequencing management method and system |
-
2021
- 2021-07-06 CN CN202110762419.0A patent/CN113590472B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011042282A1 (en) * | 2009-10-08 | 2011-04-14 | International Business Machines Corporation | Automated test execution plan generation |
| CN109783349A (en) * | 2018-12-10 | 2019-05-21 | 江苏大学 | A kind of priorities of test cases sort method and system based on dynamical feedback weight |
| CN110502447A (en) * | 2019-08-30 | 2019-11-26 | 西安邮电大学 | A kind of regression test case priority ordering method based on figure |
| CN110825620A (en) * | 2019-10-16 | 2020-02-21 | 四川大学 | Test case priority calculation method in regression test |
| CN111427802A (en) * | 2020-06-09 | 2020-07-17 | 南京大学 | Test method and system for carrying out test case priority sequencing by utilizing ensemble learning |
| CN112463636A (en) * | 2020-12-11 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Test case sequencing management method and system |
Non-Patent Citations (2)
| Title |
|---|
| AMR ABDELFATAH AHMED等: "Software testing suite prioritization using multi-criteria fitness function" * |
| 冯沈峰: "回归测试用例优先级排序问题研究" * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114978974A (en) * | 2022-05-20 | 2022-08-30 | 国网重庆市电力公司电力科学研究院 | Wireless communication module testing method, device, equipment and storage medium |
| CN114978974B (en) * | 2022-05-20 | 2023-09-19 | 国网重庆市电力公司电力科学研究院 | Wireless communication module testing method, device, equipment and storage medium |
| CN115526460A (en) * | 2022-09-09 | 2022-12-27 | 珠海安士佳电子有限公司 | Intelligent production test system for security monitoring camera |
| CN115526460B (en) * | 2022-09-09 | 2024-04-09 | 珠海安士佳电子有限公司 | Intelligent production test system for security monitoring cameras |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113590472B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113590472B (en) | Test case priority ranking method in regression test | |
| CN106376002B (en) | Management method and device and spam monitoring system | |
| CN109491914B (en) | High-impact defect report prediction method based on unbalanced learning strategy | |
| CN102541736B (en) | Acceleration test method in software reliability execution process | |
| CN109492106B (en) | Automatic classification method for defect reasons by combining text codes | |
| CN109891508A (en) | Single cell type detection method, device, equipment and storage medium | |
| CN107193732B (en) | Verification function positioning method based on path comparison | |
| CN111428733A (en) | Zero sample target detection method and system based on semantic feature space conversion | |
| CN111581086B (en) | Hybrid software error positioning method and system based on RankNet | |
| CN112184679A (en) | YOLOv 3-based wine bottle flaw automatic detection method | |
| CN115357896A (en) | Deep learning-based directional vulnerability mining method | |
| Muthusamy et al. | A test case prioritization method with weight factors in regression testing based on measurement metrics | |
| CN107102909A (en) | A kind of Fault Classification towards complex software intensive equipment system | |
| CN117725437A (en) | Machine learning-based data accurate matching analysis method | |
| CN113886373A (en) | Data processing method and device and electronic equipment | |
| CN106991171A (en) | Topic based on Intelligent campus information service platform finds method | |
| CN111026075A (en) | Error matching-based fault detection method for medium-low pressure gas pressure regulator | |
| CN113704073B (en) | Method for detecting abnormal data of automobile maintenance record library | |
| CN115422092A (en) | Software bug positioning method based on multi-method fusion | |
| CN111814909B (en) | Information processing method based on network live broadcast and online e-commerce delivery and cloud server | |
| CN115080386A (en) | Scene effectiveness analysis method and device based on automatic driving function requirement | |
| CN113792141A (en) | Feature selection method based on covariance measurement factor | |
| CN114706769A (en) | Log-based regression test-oriented black box test case sequencing method | |
| CN114186644A (en) | Defect report severity prediction method based on optimized random forest | |
| Karthik et al. | Defect association and complexity prediction by mining association and clustering rules |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |