CN113868122B - Test case screening method, system and device for regression testing - Google Patents

Test case screening method, system and device for regression testing Download PDF

Info

Publication number
CN113868122B
CN113868122B CN202111059899.0A CN202111059899A CN113868122B CN 113868122 B CN113868122 B CN 113868122B CN 202111059899 A CN202111059899 A CN 202111059899A CN 113868122 B CN113868122 B CN 113868122B
Authority
CN
China
Prior art keywords
modules
test
module combination
risk module
regression
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111059899.0A
Other languages
Chinese (zh)
Other versions
CN113868122A (en
Inventor
王少磊
郭静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Inspur Intelligent Technology Co Ltd
Original Assignee
Suzhou Inspur Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Inspur Intelligent Technology Co Ltd filed Critical Suzhou Inspur Intelligent Technology Co Ltd
Priority to CN202111059899.0A priority Critical patent/CN113868122B/en
Publication of CN113868122A publication Critical patent/CN113868122A/en
Application granted granted Critical
Publication of CN113868122B publication Critical patent/CN113868122B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a test case screening method, a system and a device for regression testing, wherein the method comprises the steps of grouping faults in a fault library according to modules to obtain fault information of each module, and dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information; all test cases of all modules in the first risk module combination are added into a regression test, the test cases of all modules in the second risk module combination are added into the regression test according to a preset proportion to form a regression test case library, and all modules in the third risk module combination do not carry out the regression test. According to the invention, based on historical test data, the modules are sequenced according to fault information, analysis processing and screening of test cases are carried out on each module, a regression test case library is formed, repeated tests on low-risk and waste cases are avoided, a large number of unnecessary test processes are avoided, test efficiency is improved, and test cost is saved.

Description

Test case screening method, system and device for regression testing
Technical Field
The invention relates to the technical field of software testing, in particular to a test case screening method, a system and a device for regression testing.
Background
At any stage in the software lifecycle, a problem may be presented to the software whenever it changes. The change in software may result from errors being discovered and modifications made, and may also be due to the addition of new modules during the integration or maintenance phase. When errors contained in the software are found, modifications to these errors may be missed if the error tracking and management system is not perfect; while the developer is not well aware of the error, the modification may be caused to correct only the external appearance of the error, without repairing the error itself, thereby causing a modification failure; modification may also produce side effects that cause new problems in the unmodified parts of the software, causing errors in functions that would otherwise work properly. Also, when a new code is added to the software, the new code may have an influence on the original code in addition to the possibility of errors contained in the new added code. Thus, whenever the software changes, we have to retest the existing functionality to determine if the modification has achieved its intended purpose, checking if the modification has compromised the original normal functionality. At the same time, new test cases need to be supplemented to test new or modified functionality. Regression testing is required to verify the correctness of the modification and its impact.
Regression testing refers to the re-testing after an old code has been modified to confirm that the modification did not introduce new errors or caused other code to produce errors. Regression testing is used as a component of the software lifecycle, and takes up a great amount of work in the whole software testing process, and multiple regression tests are performed at each stage of software development. Continuous release of new versions in progressive and rapid iterative development makes regression testing more frequent, whereas in the extreme programming approach, regression testing is more required to be performed several times per day. Therefore, it is very significant to improve the efficiency and effectiveness of regression testing by choosing the correct regression testing strategy.
The current common regression test selection mode selects all test cases in the baseline test case library to form a regression test case package, which is a safer method, and the test of all the test cases has the lowest risk of missing regression errors, but the test cost is highest, and the repeated original all the tests bring great workload.
Disclosure of Invention
The invention provides a test case screening method, a system and a device for regression testing, which are used for solving the problems of high cost and large workload of the existing regression testing.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the first aspect of the present invention provides a test case screening method for regression testing, the method comprising the steps of:
the faults in the fault library are grouped according to the modules to obtain fault information of each module, and the modules are divided into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
adding all test cases of all modules in the first risk module combination into a regression test, adding the test cases of all modules in the second risk module combination into the regression test according to a preset proportion to form a regression test case library, and not carrying out the regression test on all modules in the third risk module combination.
Further, the test cases in the regression test case library are grouped according to the attribute to obtain an automatic test case group and a non-automatic test case group, and the grouped test execution time is counted respectively.
Further, the fault information of each module includes a total fault number and a critical fault number, and the specific process of dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information is as follows:
the modules are respectively sequenced according to the sequence of the total fault quantity and the serious fault quantity from more to less;
the top a modules of the total fault quantity rank or the top a modules of the serious fault quantity rank are used as a first risk module combination;
combining the last b modules ranked by the total fault number and the last b modules ranked by the serious fault number as a third risk module;
and taking the modules except the first risk module combination and the third risk module combination as a second risk module combination.
Further, the number m of modules in the first risk module combination satisfies 0<m.ltoreq.2a, the number n of modules in the third risk module combination satisfies n.ltoreq.b, and the sum of m and n is smaller than the total number of modules.
Further, the specific process of adding the test cases of each module in the second risk module combination into the regression test according to the preset proportion is as follows:
counting the quantity ID of the cases of each module, and acquiring the effective cases and the abandoned cases in each module based on the attribute of the test cases;
and sequencing according to the sequence of the number ID of the use cases from large to small, setting the proportion of the sampling and measuring use cases of each module according to the sequencing, wherein the proportion of the sampling and measuring use cases of the module with the front sequencing is not smaller than that of the module with the rear sequencing.
Further, the ID of the sample is generated by a random number method.
Further, the modules include storage, clusters, basic input output systems, disk arrays, snapshots, and chassis management.
A second aspect of the present invention provides a test case screening system for regression testing, the system comprising:
the fault module grouping unit is used for grouping faults in the fault library according to the modules to obtain fault information of each module, and dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
the test case screening unit is used for adding all test cases of all modules in the first risk module combination into a regression test, adding the test cases of all modules in the second risk module combination into the regression test according to a preset proportion to form a regression test case library, and not carrying out the regression test on all modules in the third risk module combination.
Further, the test case screening unit includes:
the sequencing subunit is used for sequencing the modules respectively according to the order of the number of faults and the number of serious faults from more to less;
a first screening subunit, configured to use the top a modules of the ranking number of faults or the top a modules of the ranking number of serious faults as a first risk module combination;
a second screening subunit, configured to combine the last b modules ranked by the number of faults and the last b modules ranked by the number of serious faults as a third risk module;
and the third screening subunit is used for taking the modules except the first risk module combination and the third risk module combination as a second risk module combination.
The third aspect of the present invention provides a test case screening device for regression testing, which comprises a fault management system and a case management system, and the device further comprises the test case screening system, wherein the test case screening system screens test cases in the case management system by using the test case screening method based on fault information of modules in the fault management system to obtain test cases for regression testing.
The effects provided in the summary of the invention are merely effects of embodiments, not all effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
according to the invention, based on historical test data, the modules are ordered according to the number of faults and the number of serious faults, analysis processing and screening of test cases are carried out on the fault information of each module, a regression test case library is formed, repeated tests on low risk and abandoned cases are avoided, a large number of unnecessary test processes are avoided, the test efficiency is improved, and the test cost is saved.
When the test cases are extracted, proportional sampling inspection is carried out according to the number of faults of the modules, and a random extraction mode is adopted in the sampling inspection process, so that the objectivity of the extracted test cases and the accuracy of test results are ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a schematic flow chart of an embodiment of the method of the present invention;
FIG. 2 is a schematic diagram of the result of grouping modules in an embodiment of the method of the present invention;
FIG. 3 is a schematic diagram of a case-by-case spot test of a second combination of risk modules in an embodiment of the method of the present invention;
FIG. 4 is a schematic diagram of an embodiment of the system of the present invention;
fig. 5 is a schematic view of the structure of an embodiment of the device according to the present invention.
Detailed Description
In order to clearly illustrate the technical features of the present solution, the present invention will be described in detail below with reference to the following detailed description and the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different structures of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and processes are omitted so as to not unnecessarily obscure the present invention.
As shown in fig. 1, a test case screening method for regression testing according to an embodiment of the present invention includes the following steps:
s1, grouping faults in a fault library according to the modules to obtain fault information of each module, and dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
s2, adding all test cases of all modules in the first risk module combination into a regression test, adding the test cases of all modules in the second risk module combination into the regression test according to a preset proportion to form a regression test case library, and enabling all modules in the third risk module combination not to carry out the regression test.
In step S1, the modules include a storage NAS, a cluster, a BIOS, a RAID, a snapshot, and chassis management. The attributes of the fault include the module to which it belongs, the severity and the fault ID.
The fault information of the modules comprises total fault quantity and serious fault quantity, and the specific processes of dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information are as follows:
the modules are respectively sequenced according to the sequence of the total fault quantity and the serious fault quantity from more to less; the top a modules of the total fault quantity rank or the top a modules of the serious fault quantity rank are used as a first risk module combination; combining the last b modules ranked by the total fault number and the last b modules ranked by the serious fault number as a third risk module; and taking the modules except the first risk module combination and the third risk module combination as a second risk module combination.
The number m of the modules in the first risk module combination meets 0<m and is less than or equal to 2a, the number n of the modules in the third risk module combination meets n and is less than or equal to b, and the sum of m and n is smaller than the total number of the modules.
As shown in fig. 2, an exemplary illustration of the modules being ordered by number of failures (BUGs), a first risk module combination (NAS, network Attached Storage, network attached storage with 100 BUGs) was noted as a high risk module, all tested, a third risk module combination (basic input output system BIOS with 10 BUGs) was noted as a low risk module, no test was done, and a spot test strategy was formulated for a second risk module combination (disk array RAID,60 BUGs; cluster 30 BUGs).
As shown in fig. 3, the specific process of adding the test cases of each module in the second risk module combination into the regression test according to the preset proportion is as follows:
s21, counting the number of cases ID of each module to form an array { (module Q, BUG number, maximum case ID Q), … … }, acquiring effective cases and abandoned cases in each module based on the attribute of the test cases, and respectively counting the IDs of abandoned cases under each module to form respective abandoned case arrays of each module, such as modules Q { ID1, ID2, ID3 … }. The attributes of the test cases include the case ID, whether the test cases are abandoned, whether the test cases are automated and the single-round test duration. There are 100 test cases in the raid module in fig. 3, where the database obsolete case 10 valid cases 90.
S22, sorting according to the sequence of the number ID of the use cases from large to small, setting the proportion of the sampling and measuring use cases of each module according to the sorting, wherein the proportion of the sampling and measuring use cases of the module with the front sorting is not smaller than that of the module with the rear sorting.
Assuming that the number of modules in the second risk module combination is x, setting the module sampling test case of x/3 before the failure number is clapped to be 50%, setting the module sampling test case of the middle x/3 to be 25%, and finally setting the module sampling test case of x/3 to be 5%.
In addition, an example of the sampling ratio is given in fig. 3, specifically, for the raid module, 50% of the sampling effective cases; for the snapshot module, 25% of the active cases are extracted and tested; for the chassis management module, the active cases are sampled 25%, and for the cluster module, the active cases are sampled 5%.
The ID of the sample is generated by random number method. For example, the maximum ID of the use case is Q, and according to the order of the number of the BUGs of the module, 50% of the use case needs to be sampled, Q0.5 random numbers can be sampled from [1, Q ] by using a random number algorithm, and put into an array D { n1, n2, n3 … }, during which, each random number is generated, whether the random number exists in D needs to be judged before the random number is put into the array D, if the random number exists, the data is not put into the array in the current sampling, if the random number does not exist, whether the random number exists in the corresponding module waste ID array needs to be judged, if the random number exists, the random number is not put into the array in the current sampling, and if the random number does not exist, the random number is generated and put into the array D. Finally, the use cases corresponding to the use case IDs in the random array D corresponding to each module are put into a regression testing range. Thus, the randomness of the extraction of the use cases can be ensured, and the extracted use cases can be ensured to be available in the use case library.
On the basis of the above embodiment, the embodiment of the present invention further includes the steps of:
and grouping the test cases in the regression test case library according to the attribute to obtain an automatic test case group and a non-automatic test case group, and respectively counting the grouped test execution time. The workload required by the rapid bottle cap regression test is convenient, and the workload can be used as a reference basis for modifying the screening strategy.
As shown in fig. 4, a test case screening system for regression testing according to an embodiment of the present invention includes a fault module grouping unit 1 and a test case screening module 2.
The fault module grouping unit 1 is used for grouping faults in a fault library according to the modules to obtain fault information of each module, and dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
the test case screening unit 2 is configured to add all test cases of each module in the first risk module combination into a regression test, add test cases of each module in the second risk module combination into the regression test according to a preset proportion, and form a regression test case library, where each module in the third risk module combination does not perform the regression test.
The test case screening unit 2 includes a sorting subunit 21, a first screening subunit 22, a second screening subunit 23, and a third screening subunit 24.
The sequencing subunit 21 is configured to sequence the modules according to the order of the number of faults and the number of serious faults from more to less; the first screening subunit 22 is configured to rank the top a modules of the failure number rank the top a or the serious failure number rank the top a modules as a first risk module combination; the second screening subunit 23 is configured to use the last b modules ranked by the number of faults and the last b modules ranked by the number of serious faults as a third risk module combination; the third screening subunit 24 is configured to use a module other than the first risk module combination and the third risk module combination as the second risk module combination.
As shown in fig. 4, the test case screening device for regression testing provided by the embodiment of the invention comprises a fault management system, a case management system and a test case screening system, wherein the test case screening system screens test cases in the case management system based on fault information of modules in the fault management system to obtain the test cases for regression testing.
The classification of the modules in the fault management system is always the same as the classification of the modules in the use case management system, so that the interaction of the information of the modules is facilitated.
While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.

Claims (8)

1. A test case screening method for regression testing, the method comprising the steps of:
the faults in the fault library are grouped according to the modules to obtain fault information of each module, and the modules are divided into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
adding all test cases of all modules in the first risk module combination into a regression test, adding the test cases of all modules in the second risk module combination into the regression test according to a preset proportion to form a regression test case library, wherein all modules in the third risk module combination do not perform the regression test;
the fault information of each module comprises total fault quantity and serious fault quantity, and the specific process of dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information is as follows:
the modules are respectively sequenced according to the sequence of the total fault quantity and the serious fault quantity from more to less;
the top a modules of the total fault quantity rank or the top a modules of the serious fault quantity rank are used as a first risk module combination;
combining the last b modules ranked by the total fault number and the last b modules ranked by the serious fault number as a third risk module;
and taking the modules except the first risk module combination and the third risk module combination as a second risk module combination.
2. The test case screening method for regression testing according to claim 1, wherein the test cases in the regression test case library are grouped according to attributes to obtain an automated test case group and a non-automated test case group, and the grouped test execution durations are counted respectively.
3. The test case screening method for regression testing according to claim 1, wherein the number of modules m in the first risk module combination satisfies 0<m.ltoreq.2a, the number of modules n in the third risk module combination satisfies n.ltoreq.b, and the sum of m and n is smaller than the total number of modules.
4. The test case screening method for regression testing according to claim 1, wherein the specific process of adding the test cases of each module in the second risk module combination into the regression testing according to the preset ratio is as follows:
counting the quantity ID of the cases of each module, and acquiring the effective cases and the abandoned cases in each module based on the attribute of the test cases;
and sequencing according to the sequence of the number ID of the use cases from large to small, setting the proportion of the sampling and measuring use cases of each module according to the sequencing, wherein the proportion of the sampling and measuring use cases of the module with the front sequencing is not smaller than that of the module with the rear sequencing.
5. The test case screening method for regression testing according to claim 4, wherein the ID of the sample is generated by using a random number method.
6. The test case screening method for regression testing of claim 1 or 2, wherein the modules include storage, clusters, basic input output system, disk arrays, snapshots, and chassis management.
7. A test case screening system for regression testing, the system comprising:
the fault module grouping unit is used for grouping faults in the fault library according to the modules to obtain fault information of each module, and dividing the modules into a first risk module combination, a second risk module combination and a third risk module combination based on the fault information;
the test case screening unit is used for adding all test cases of all modules in the first risk module combination into a regression test, adding the test cases of all modules in the second risk module combination into the regression test according to a preset proportion to form a regression test case library, and enabling all modules in the third risk module combination not to perform the regression test;
the test case screening unit includes:
the sequencing subunit is used for sequencing the modules respectively according to the order of the number of faults and the number of serious faults from more to less;
a first screening subunit, configured to use the top a modules of the ranking number of faults or the top a modules of the ranking number of serious faults as a first risk module combination;
a second screening subunit, configured to combine the last b modules ranked by the number of faults and the last b modules ranked by the number of serious faults as a third risk module;
and the third screening subunit is used for taking the modules except the first risk module combination and the third risk module combination as a second risk module combination.
8. The test case screening device for the regression test comprises a fault management system and a case management system, and is characterized by further comprising the test case screening system according to claim 7, wherein the test case screening system screens test cases in the case management system by using the test case screening method according to claim 1 or 2 based on fault information of modules in the fault management system to obtain test cases for the regression test.
CN202111059899.0A 2021-09-10 2021-09-10 Test case screening method, system and device for regression testing Active CN113868122B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111059899.0A CN113868122B (en) 2021-09-10 2021-09-10 Test case screening method, system and device for regression testing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111059899.0A CN113868122B (en) 2021-09-10 2021-09-10 Test case screening method, system and device for regression testing

Publications (2)

Publication Number Publication Date
CN113868122A CN113868122A (en) 2021-12-31
CN113868122B true CN113868122B (en) 2023-11-14

Family

ID=78995155

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111059899.0A Active CN113868122B (en) 2021-09-10 2021-09-10 Test case screening method, system and device for regression testing

Country Status (1)

Country Link
CN (1) CN113868122B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105528293A (en) * 2015-12-14 2016-04-27 苏州洞察云信息技术有限公司 System and method for selecting test case for regression test
CN111274126A (en) * 2020-01-14 2020-06-12 华为技术有限公司 Test case screening method, device and medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10002216B2 (en) * 2015-12-02 2018-06-19 Cavium, Inc. Systems and methods for dynamic regression test generation using coverage-based clustering

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105528293A (en) * 2015-12-14 2016-04-27 苏州洞察云信息技术有限公司 System and method for selecting test case for regression test
CN111274126A (en) * 2020-01-14 2020-06-12 华为技术有限公司 Test case screening method, device and medium

Also Published As

Publication number Publication date
CN113868122A (en) 2021-12-31

Similar Documents

Publication Publication Date Title
Cotroneo et al. Fault triggers in open-source software: An experience report
Abreu et al. A practical evaluation of spectrum-based fault localization
CN102214139B (en) Automatic test performance control and debugging method facing distributed system
US20090265681A1 (en) Ranking and optimizing automated test scripts
US20130111267A1 (en) Optimizing regression testing based on code coverage analysis
CN110008129B (en) Reliability test method, device and equipment for storage timing snapshot
Goseva-Popstojanova et al. Large empirical case study of architecture-based software reliability
US20170123873A1 (en) Computing hardware health check
US20140122928A1 (en) Network Processor Online Logic Test
CN110515843B (en) Test case priority ordering method based on defect set and inverted index
US20210311918A1 (en) Computer system diagnostic log chain
CN109684204A (en) A kind of http interface automatic test approach
CN112019404A (en) Method and device for automatically testing FC-AE-1553 communication protocol chip
CN115080389A (en) Test system, method, equipment and storage medium for improving index statistical efficiency
CN114996127A (en) Intelligent test method and system for solid state disk firmware module
CN101894073B (en) Defect automatic positioning device based on control flow intersection and automatic positioning method thereof
US20100251029A1 (en) Implementing self-optimizing ipl diagnostic mode
CN113868122B (en) Test case screening method, system and device for regression testing
US7904754B2 (en) Systems and methods for automated determination of out of memory handling
CN116090380B (en) Automatic method and device for verifying digital integrated circuit, storage medium and terminal
CN112731117A (en) Automatic verification method and system for chip, and storage medium
US11715545B2 (en) In-system test of a memory device
CN113282496B (en) Automatic interface testing method, device, equipment and storage medium
CN112486748B (en) Test system and test method thereof
CN112527584A (en) Software efficiency improving method and system based on script compiling and data acquisition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant