CN112783800B - Test case screening method and device - Google Patents
Test case screening method and device Download PDFInfo
- Publication number
- CN112783800B CN112783800B CN202110296587.5A CN202110296587A CN112783800B CN 112783800 B CN112783800 B CN 112783800B CN 202110296587 A CN202110296587 A CN 202110296587A CN 112783800 B CN112783800 B CN 112783800B
- Authority
- CN
- China
- Prior art keywords
- case
- test
- coverage rate
- test case
- code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/368—Test management for test version control, e.g. updating test cases to a new software version
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/9035—Filtering based on additional data, e.g. user or group profiles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention provides a test case screening method and a test case screening device, which can be applied to the technical field of computer application and are used for acquiring a change code; obtaining a test case to be selected; determining the increment coverage rate of the test case to be selected for the changed code, wherein the increment coverage rate represents the coverage rate of the test case to be selected for the changed code when the test case to be selected is executed; and when the increment coverage rate meets the preset condition, adding the to-be-selected test case into the case library. The use case library can be reduced, and the test efficiency is improved.
Description
Technical Field
The present invention relates to the field of computer application technologies, and in particular, to a test case screening method and apparatus.
Background
The test cases are important basis in the iteration process of the software development project, a large number of test cases can be accumulated to a case library along with the rapid iteration and development of the service, and the current test cases need to be subjected to full-scale regression when certain services are reconstructed. For example, in the process of modifying the old code corresponding to the old version in the process of developing the new version, regression testing is required for the modified code, and the test is performed again by using all accumulated test cases to confirm that no new error is introduced in the modification or that other codes generate errors, etc. The large number of test cases in the case library is definitely a huge workload for testing, the execution time required for testing is also not trivial, and how to guarantee the verification range with fewer test cases is an important content in the testing process.
Disclosure of Invention
The embodiment of the invention aims to provide a test case screening method and device for screening test cases to be added to a case library, reducing the case library and improving the test efficiency. The specific technical scheme is as follows:
in a first aspect of the present invention, there is provided a test case screening method, including:
acquiring a change code;
obtaining a test case to be selected;
determining the increment coverage rate of the test case to be selected for the change code, wherein the increment coverage rate represents the coverage rate of the change code when the test case to be selected is executed;
and when the increment coverage rate meets a preset condition, adding the test case to be selected to a case library.
Optionally, when the incremental coverage rate meets a preset condition, adding the test case to be selected to a case library, including:
judging whether the increment coverage is larger than a first preset coverage or not;
and when the increment coverage rate is larger than a first preset coverage rate, adding the to-be-selected test case to a case library.
Optionally, when the incremental coverage rate meets a preset condition, adding the test case to be selected to a case library, including:
Calculating the case repetition degree based on the increment coverage and a second preset coverage;
and when the repetition degree of the use cases is smaller than the preset repetition degree, adding the to-be-selected test use cases into the use case library.
Optionally, the test cases to be selected include test cases to be selected with different priority levels, where the priority levels are determined according to service requirements;
the determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
determining a first coverage rate of the change code when executing the highest-level to-be-selected test case, wherein the highest-level to-be-selected test case is the test case with the highest priority level in the to-be-selected test cases;
and when the increment coverage rate meets a preset condition, adding the test case to be selected to a case library, wherein the method comprises the following steps:
calculating a case repetition degree based on the first coverage rate and the basic coverage rate;
judging whether the repetition degree of the use case is smaller than a preset repetition degree or not;
when the case repetition degree is smaller than the preset repetition degree, adding the highest-level to-be-selected test case to the case library;
the determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
Determining a second coverage rate of the change codes when executing the test cases according to the order of the priority level from high to low for each of other test cases to be selected; the other test cases to be selected comprise test cases except the highest-level test case to be selected in the test cases to be selected;
and when the increment coverage rate meets a preset condition, adding the test case to be selected to a case library, wherein the method comprises the following steps:
judging whether the second coverage rate is larger than the coverage rate before executing the test case;
and when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library.
Optionally, after the determining whether the case repetition degree is less than a preset repetition degree, the method further includes:
when the repetition degree of the use cases is not less than the preset repetition degree, modifying the highest-level to-be-selected test use cases; and returning to the step of determining the first coverage rate of the changed codes when the highest-level candidate test case is executed.
Optionally, the calculating the case repetition based on the first coverage rate and the base coverage rate includes:
Determining the current coverage rate of the change code when executing the test case according to each test case in the highest-level to-be-selected test cases in sequence;
when the current coverage rate is the same as the coverage rate before the test case is executed, adding 1 to the repetition number of the case until all the test cases in the highest-level to-be-selected test case are executed, and obtaining the final repetition number of the case;
and taking the ratio of the final use case repetition number to the total number of use cases as the use case repetition degree, wherein the total number of use cases is the total number of the highest-level to-be-selected test use cases.
Optionally, the acquiring the change code includes:
acquiring a code to be tested;
searching a starting node and an ending node in the code to be detected;
obtaining a change file according to the starting node and the ending node;
analyzing and obtaining a code implementation method corresponding to the change file through a source code analysis tool, and calculating a method parameter value corresponding to the change file;
if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, taking the method realized by the code corresponding to the change file as a change method;
The determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
analyzing the change method through a coverage rate analysis tool to generate a coverage rate report, wherein the coverage rate report comprises the increment coverage rate of the test case to be selected for the change method.
In a second aspect of the present invention, there is also provided a test case screening apparatus, including:
the acquisition module is used for acquiring the change code; the change code includes a code updated with respect to a reference code; obtaining a test case to be selected; the test cases to be selected are the test cases to be added to the case library, and the test cases are used for testing codes;
the determining module is used for determining the increment coverage rate of the test case to be selected for the change code, wherein the increment coverage rate represents the coverage rate of the test case to be selected for the change code when the test case to be selected is executed.
The adding module is used for adding the to-be-selected test cases to a case library if the increment coverage rate meets a preset condition; otherwise, the test case to be selected is not added to the case library.
Optionally, the adding module is specifically configured to determine whether the incremental coverage rate is greater than a first preset coverage rate; and when the increment coverage rate is larger than a first preset coverage rate, adding the to-be-selected test case to a case library.
Optionally, the adding module is specifically configured to calculate a case repetition rate based on the incremental coverage rate and a second preset coverage rate; and when the repetition degree of the use cases is smaller than the preset repetition degree, adding the to-be-selected test use cases into the use case library.
Optionally, the test cases to be selected include test cases to be selected with different priority levels, where the priority levels are determined according to service requirements;
the determining module is specifically configured to determine a first coverage rate of the change code when the highest-level to-be-selected test case is executed, where the highest-level to-be-selected test case is a test case with a highest priority level in the to-be-selected test cases;
the adding module is specifically configured to calculate a use case repetition rate based on the first coverage rate and the basic coverage rate; judging whether the repetition degree of the use case is smaller than a preset repetition degree or not; when the case repetition degree is smaller than the preset repetition degree, adding the highest-level to-be-selected test case to the case library; when the repetition degree of the case is not less than the preset repetition degree, modifying the highest-level to-be-selected test case; returning to the step of determining the first coverage rate of the changed codes when the highest-level to-be-selected test case is executed;
The determining module is specifically configured to determine, for each test case in other test cases to be selected, a second coverage rate of the change code when the test case is executed according to a sequence from high priority to low priority; the other test cases to be selected comprise test cases except the highest-level test case to be selected in the test cases to be selected;
the adding module is specifically configured to determine whether the second coverage rate is greater than a coverage rate before the test case is executed; when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library; otherwise, the test case is not added to the case library.
Optionally, the adding module is specifically configured to determine, for each test case in the highest-level to-be-selected test cases in turn, a current coverage rate of the change code when the test case is executed; when the current coverage rate is the same as the coverage rate before the test case is executed, adding 1 to the repetition number of the case until all the test cases in the highest-level to-be-selected test case are executed, and obtaining the final repetition number of the case; and taking the ratio of the final use case repetition number to the total number of use cases as the use case repetition degree, wherein the total number of use cases is the total number of the highest-level to-be-selected test use cases.
Optionally, the acquiring module is specifically configured to acquire a code to be tested; searching a starting node and an ending node in the code to be detected; obtaining a change file according to the starting node and the ending node; analyzing and obtaining a code implementation method corresponding to the change file through a source code analysis tool, and calculating a method parameter value corresponding to the change file; if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, taking the method realized by the code corresponding to the change file as a change method;
the determining module is specifically configured to parse the change method through a coverage rate analysis tool, and generate a coverage rate report, where the coverage rate report includes an incremental coverage rate of the test case to be selected for the change method.
In still another aspect of the present invention, there is provided a test case screening apparatus, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and a processor, configured to implement the method steps described in the first aspect when executing the program stored in the memory.
In a further aspect of the present invention, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the method steps of the first aspect.
The test case screening method and device provided by the embodiment of the invention acquire the change codes; the change code includes a code updated with respect to a reference code; obtaining a test case to be selected; the test cases to be selected are the test cases to be added to the case library, and the test cases are used for testing codes; determining the increment coverage rate of the test case to be selected for the changed code, wherein the increment coverage rate represents the coverage rate of the test case to be selected for the changed code when the test case to be selected is executed; if the increment coverage rate meets the preset condition, adding the to-be-selected test case into a case library; otherwise, the test case to be selected is not added to the case library. The increment coverage rate of the test cases to be selected for the variable codes can be determined, namely the coverage rate of the test cases to be selected for the variable codes is determined when the increment coverage rate meets the preset conditions, and then the test cases to be selected are added to the case library, so that the test cases to be added to the case library can be screened, the case library is reduced, and the test efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flowchart of a test case screening method according to an embodiment of the present invention;
FIG. 2 is a flow chart of acquiring a change code according to an embodiment of the present invention;
FIG. 3 is a flow chart of calculating delta coverage from method dimensions in an embodiment of the invention;
FIG. 4 is another flowchart of a test case screening method according to an embodiment of the present invention;
FIG. 5 is an application diagram of a test case screening method according to an embodiment of the present invention;
FIG. 6 is a flow chart of calculating the repetition rate of a use case according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a test case screening device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of test case screening apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.
The inventor researches a large number of service case libraries to find that most service lines only merge the existing test cases into the case libraries after the function verification is passed, and the test cases cannot be subjected to secondary processing, so that a large amount of redundancy is generated in the case libraries, the base number of the case libraries is increased, and the test efficiency is lower.
According to the embodiment of the invention, the historical test cases and/or newly added test cases and the like are screened by selecting the reasonable standard, and the test cases are effectively put in storage, so that the minimization of the test case set can be ensured, more effective candidate cases are provided for testing, and the testing efficiency is further improved. According to the embodiment of the invention, the coverage rate generated by the test cases is introduced, and the coverage rate is used as a judging standard for whether to put the test cases in storage or not, namely, the test cases added into the case library are screened based on the coverage rate, so that the screening of historical test cases and/or newly added test cases and the like is realized, the case library is reduced, the test efficiency is improved, the minimum set storage is expected to be selected, the minimization of the test case library is ensured, and the test efficiency is improved.
The test case screening method provided by the embodiment of the invention can be applied to business test, effectively locks the test range, ensures the validity of the test case, simultaneously minimizes the test case set and improves the return test efficiency. For example, in a scenario where the service development time is long, the functions are multiple, the use cases are multiple, and the test efficiency needs to be improved and the service test is performed by a company that simplifies the use cases.
The test case screening method provided by the embodiment of the invention is described in detail below.
The embodiment of the invention provides a test case screening method, which can comprise the following steps:
acquiring a change code;
obtaining a test case to be selected;
determining the increment coverage rate of the test case to be selected for the changed code, wherein the increment coverage rate represents the coverage rate of the test case to be selected for the changed code when the test case to be selected is executed;
and when the increment coverage rate meets the preset condition, adding the to-be-selected test case into the case library.
In the embodiment of the invention, the increment coverage rate of the test cases to be selected for the change codes is determined, namely the coverage rate of the test cases to be selected for the change codes is determined when the increment coverage rate meets the preset condition, and then the test cases to be selected are added to the case library, so that the test cases to be added to the case library can be screened, the case library is reduced, and the test efficiency is improved.
The execution subject of the test case screening method provided by the embodiment of the invention can be electronic equipment. In particular, the electronic device may include a terminal, a processor, a server, and the like.
FIG. 1 is a flowchart of a test case screening method according to an embodiment of the present invention. Referring to fig. 1, the test case screening method provided by the embodiment of the present invention may include:
S101, acquiring a change code.
Altering code may also be understood as updating code relative to a reference code, may include adding code, such as code to fulfill new requirements, and/or modifying code, etc.
S102, obtaining a test case to be selected.
The test cases to be selected can comprise original historical test cases in a case library, and also can comprise newly added test cases, such as test cases written according to a requirement document when a new requirement is required to be tested; the history test case and the newly added test case can be simultaneously included.
S103, determining the increment coverage rate of the test case to be selected for the variable code.
The increment coverage rate represents the coverage rate of the change codes when the test case to be selected is executed.
It is simply understood that the incremental coverage may reflect the extent to which the test case to be selected covers the change code, such as whether executing the test case to be selected can cover the change code, the proportion of the covered code to the change code, and so on.
Specifically, the change code may be parsed by a coverage analysis tool to generate a coverage report.
The coverage report may include incremental coverage of the test case candidate for the change code.
In one implementation, the coverage analysis tool may be jacaco. Jacaco is an open source coverage tool. Jacoco can obtain coverage counters, coverage counter row coverage, class coverage, method coverage, branch coverage, instruction coverage, and ring complexity.
Specifically, jacaco executes the test cases to test, exec files (coverage rate information files) are generated in the test process, the exec files are analyzed, analysis is only carried out on the changed codes through a visual method of a ClassProbesAldapter class, and other non-designated classes and methods are not processed, so that a coverage rate report comprising the increment coverage rate of the test cases to be selected for the changed codes is obtained.
S104, when the increment coverage rate meets the preset condition, adding the to-be-selected test case into the case library.
Whether the test case to be selected is covered to the change code or not can be indicated by the increment coverage, or coverage paths of a plurality of test cases to be selected are the same, and the like.
If the to-be-selected test case is covered to the change code, the to-be-selected test case can be added to the case library as an effective test case, otherwise, the to-be-selected test case is considered as an ineffective test case.
If the coverage rate paths of the multiple test cases to be selected are the same, the multiple test cases to be selected with the same coverage rate paths can be generalized into one test case, and the generalized test case is added to the case library, so that the number of the test cases in the case library can be reduced, and the return speed can be improved. The coverage path may be understood as a specific code in the coverage change code of the test case to be selected, such as the 1 st line to the 10 th line codes.
In an alternative embodiment, S104 may include:
judging whether the increment coverage rate is larger than a first preset coverage rate, and adding the to-be-selected test case to the case library when the increment coverage rate is larger than the first preset coverage rate.
In one manner, the first preset coverage rate may be a preset value determined according to an actual demand or an empirical value. In another approach, the first predetermined coverage is a coverage calculated based on existing test cases in the case library.
The method is simple, only needs to judge whether the increment coverage is larger than the first preset coverage, and when the increment coverage is larger than the first preset coverage, the test cases to be selected are added to the case library. If only the code covered by the test case to be selected is required to be judged to be more than the code covered by the existing test case, the test cases to be added to the case library can be screened in a relatively simple mode.
In another alternative embodiment, S104 may include:
calculating the case repetition based on the increment coverage and the second preset coverage; and when the repetition degree of the use cases is smaller than the preset repetition degree, adding the test cases to be tested to the use case library.
In one manner, the second preset coverage rate may be a preset value determined according to an actual demand or an empirical value. In another approach, the second predetermined coverage is a coverage calculated based on existing test cases in the case library.
The case repetition degree may represent a degree of repetition between a plurality of test cases newly added. Alternatively, the case repetition degree may represent the degree of repetition between the newly added test case and the test case existing in the case library. The degree of repetition may also be understood as the degree of repetition of multiple test case coverage paths.
The test cases to be added to the case library are screened through the repeatability, so that the repetition, namely redundancy, among the test cases in the case library can be further reduced, and the test cases can be further reduced.
In the embodiment of the invention, the increment coverage rate of the test case to be selected for the changed code can be determined, namely, the coverage rate of the test case to be selected for the changed code is performed, and the test case to be selected is added to the case library when the increment coverage rate meets the preset condition, so that the test case to be added to the case library can be screened, the case library is reduced, and the test efficiency is improved. Based on the increment coverage as a judging standard for judging whether to put test cases into storage, namely, based on the increment coverage, screening the test cases added into the case library, the minimum set storage can be selected to the greatest extent, the minimization of the test case library is ensured, and the test efficiency is improved.
In an alternative embodiment, as shown in fig. 2, S101: acquiring the change code may include:
s1011, obtaining a code to be tested.
The code to be tested may be code submitted by a developer to be tested.
S1012, searching a start node and an end node in the code to be detected.
S1013, obtaining a change file according to the start node and the end node.
In writing code, each time the code is updated, such as when the code is modified or added, the updated nodes, such as the start node and the end node, are typically marked.
The starting node and ending node in the branch may be looked up based on the developer's lifting code, such as the value of SHA-1 (string computed by the secure hash algorithm) in the code management tool Git.
With the Git Api (code management tool interface), a changed file, i.e., a changed file, is calculated from the SHA-1 value, and a non-java file, a test file, and the like are excluded.
S1014, analyzing the method of obtaining the code implementation corresponding to the change file through the source code analysis tool, and calculating the method parameter value corresponding to the change file.
S1015, if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, the method implemented by the code corresponding to the change file is taken as a change method.
The source code analysis tool may be a java development tool (Java Development Tools, JDT). The method parameter value may be an md5 (message digest algorithm) value.
And analyzing by using a JDT (Java integrated development environment plug-in with complete functions) platform to obtain a code implementation method corresponding to the change file, calculating a method parameter value corresponding to the change file, calculating a method for implementing the code to be tested, and calculating a method parameter value corresponding to the code to be tested. Specifically, JDT is used to analyze the source code and analyze the md5 value of the method for calculating the new and old (before and after the change), compare whether the md5 values before and after the change are the same, if so, indicate that there is a change, and take the method implemented by the code corresponding to the changed file as the change method.
S103: the determining the incremental coverage of the test case to be selected for the variant code may include:
analyzing the change method through a coverage rate analysis tool to generate a coverage rate report, wherein the coverage rate report comprises the increment coverage rate of the test case to be selected for the change method.
The coverage analysis tool may be jacaco. Specifically, jacaco executes the test cases to test, generates exec files in the test process, analyzes the exec files, analyzes only the change method by a visual method of the class ClassProbesAlyptor, and obtains coverage reports including the incremental coverage of the test cases to be selected for the change method without processing other non-specified classes and methods.
FIG. 3 is a flow chart of calculating delta coverage from method dimensions in an embodiment of the invention, as shown in FIG. 3, may include the following process.
Firstly, acquiring a test code; and acquiring the starting node and the ending node, namely, the starting node and the ending node in the lifting code searching branch.
And then, obtaining a change file according to the starting node and the ending node.
According to the embodiment of the invention, the method of changing is obtained through JDT analysis, so that the non-java files are removed before analysis, and then the method of changing is obtained through JDT analysis.
Meanwhile, the tested service is deployed; jacoco service deployment; and generating exec files based on the deployed tested service and the Jacoco service execution test cases.
Finally, calculating the coverage rate of the change method based on the change method and the exec file.
The coverage rate calculation range can be enlarged to be the coverage rate of the change method by avoiding focusing on excessive compaction caused by line code change, namely the incremental coverage rate is calculated from the method dimension in the embodiment of the invention.
In an alternative embodiment, the test cases to be selected may include test cases to be selected with different priority levels, where the priority levels are determined according to the service requirements.
The different priority levels may include P0, P1, P2, P3, P4, etc., with the priority level of P0→P1→P2→P3→P4 being from high to low.
FIG. 4 is another flowchart of a test case screening method according to an embodiment of the present invention. Referring to fig. 4, S103: the determining the incremental coverage of the test case to be selected for the variant code may include:
s1031, determining a first coverage rate of the change codes when executing the highest-level candidate test case.
The highest-level test case to be selected is the test case with the highest priority level in the test cases to be selected.
For example, the candidate test case with the highest priority level may be the test case with the P0 priority level. For the existing test cases in the case library, a quality assurance (Quality Assurance, QA) system screens the test cases of the most core P0 level according to the service line requirements. Or, when testing the new demand, the QA system writes the test cases with P0 priority level according to the demand document.
Specifically, the process of determining the first coverage of the altered code when executing the highest-level candidate test case may refer to the process of executing S103 in the above-described embodiment.
S104: when the increment coverage meets the preset condition, adding the test case to be selected to the case library may include:
s1041, calculating the repetition rate of the use case based on the first coverage rate and the basic coverage rate.
The basic coverage rate can be a preset value determined according to an empirical value or actual requirements; the coverage may be calculated based on the test cases existing in the case library.
S1042, judging whether the repetition degree of the use case is smaller than the preset repetition degree.
When the case repetition degree is smaller than the preset repetition degree, step S1043 is performed.
The preset repeatability can be determined according to actual requirements or experience, for example, 10%.
S1043, adding the highest-level candidate test case to the case library.
In one implementation, when the repetition degree of the case is not less than the preset repetition degree, modifying the highest-level to-be-selected test case; and returns to the step of executing the first coverage rate of the change code when determining to execute the highest-level candidate test case, that is, returns to S1031.
S103: the determining the incremental coverage of the test case to be selected for the variant code may include:
s1032, determining a second coverage rate of the change codes when executing the test cases according to the order of the priority level from high to low for each of the other test cases to be selected.
The other test cases to be selected include test cases except the highest-level test case among the test cases to be selected.
For example, according to the order of the priority levels of the test cases from high to low, for example, the test cases of P1→P2→P3→P4 are executed one by one, and the second coverage rate corresponding to each test case except the test case to be selected at the highest level in the test cases to be selected is determined.
S104: when the increment coverage meets the preset condition, adding the test case to be selected to the case library may include:
s1044, judging whether the second coverage rate is larger than the coverage rate before executing the test case.
S1045, when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library.
If the corresponding increment coverage rate of the test case after execution is increased, adding the test case into a case library; if the corresponding increment coverage rate is not increased, the test case is repeated with the previous test case, and the test case is regarded as an invalid case, namely, the test case is not put in storage.
In a specific example, fig. 5 is an application schematic diagram of a test case screening method according to an embodiment of the present invention, and as shown in fig. 5, the method may include the following procedures.
Aiming at the test case of the P0 level, (1) acquiring basic coverage rate data and repetition rate; (2) if the repetition rate is less than 10%, adding the test cases into a case library; (3) if the repetition rate is not less than 10%, the test cases of the P0 level are manually adjusted, for example, the test cases of the P0 level are modified, or the test cases of the P0 level are selected again.
Aiming at the test cases of the non-P0 level, (4) executing the test cases of the non-P0 level one by one; (5) the coverage rate is increased, and the test cases are added into a case library; (6) and if the coverage rate is unchanged, removing the case, and not warehousing the test case.
The basic coverage rate data may be a first coverage rate of the change code when the highest-level candidate test case is executed, which is determined in S1031.
The repetition rate may be the use case repetition rate calculated based on the first coverage rate and the base coverage rate in S1041.
The first coverage rate of the change code when the test case is to be selected at the highest level determined in S1031 is used as basic data, that is, when the test case at the non-P0 level is to be executed for the first time, the increment coverage rate generated by the test case at the non-P0 level is compared with the first coverage rate, if the increment coverage rate generated by the test case at the non-P0 level is greater than the first coverage rate, the test case at the non-P0 level is added to the case library, the test case at the non-P0 level is executed for the first time, the test case at the non-P0 level is executed again, the increment coverage rate generated by the test case at the 2 nd is compared with the test case at the non-P0 level, if the increment coverage rate generated by the test case at the 2 nd is greater than the test case at the non-P0 level is executed for the first time, the test case library is added to the test case at the 3 rd test case at the non-P0 level is executed again, and the test case at the non-P0 level is executed in sequence until the test case at the non-P0 level is executed for the fourth test case.
In an alternative embodiment, S1041: calculating the use case repetition based on the first coverage and the base coverage, as shown in fig. 6, may include:
s601, determining the current coverage rate of the change code when executing the test case according to each test case in the highest-level candidate test cases.
S602, when the current coverage rate is the same as the coverage rate before the test case is executed, adding 1 to the repetition number of the case until all test cases in the highest-level to-be-selected test case are executed, and obtaining the final repetition number of the case.
S603, taking the ratio of the final use case repetition number to the total number of use cases as the use case repetition degree.
Wherein the total number of cases is the total number of the test cases to be selected at the highest level.
Specifically, the total number of cases is m, if the current increment coverage rate is unchanged after the test cases are executed, the number n of case repetition is increased by 1, and after all the test cases are executed, the repetition rate (r) of the current case set is calculated by the following formula:
according to the embodiment of the invention, the increment coverage is calculated through a method level increment coverage calculation method, namely the increment coverage is calculated from the method dimension, the use case repetition degree concept is introduced, and the increment coverage and the use case repetition degree are used as the standard for screening the use case set, namely the standard for screening the test use cases added into the use case library. The increment coverage rate and the case repetition rate are introduced in the case archiving stage, repeated test cases can be rapidly removed, the test case set is guaranteed to be minimized, and the function test back efficiency is improved.
Corresponding to the test case screening method provided in the foregoing embodiment, an embodiment of the present invention provides a test case screening device, as shown in fig. 7, which may include:
an acquisition module 701, configured to acquire a change code; the change code includes a code updated with respect to a reference code; obtaining a test case to be selected; the test cases to be selected are the test cases to be added to the case library, and the test cases are used for testing codes;
the determining module 702 is configured to determine an incremental coverage of the test case to be selected for the change code, where the incremental coverage represents a coverage of the change code when the test case to be selected is executed.
An adding module 703, configured to add the test case to be selected to a case library if the incremental coverage meets a preset condition; otherwise, the test case to be selected is not added to the case library.
Optionally, the adding module 703 is specifically configured to determine whether the incremental coverage is greater than a first preset coverage; and when the increment coverage rate is larger than a first preset coverage rate, adding the to-be-selected test case to a case library.
Optionally, the adding module 703 is specifically configured to calculate a case repetition rate based on the incremental coverage rate and a second preset coverage rate; and when the repetition degree of the use cases is smaller than the preset repetition degree, adding the to-be-selected test use cases into the use case library.
Optionally, the test cases to be selected include test cases to be selected with different priority levels, where the priority levels are determined according to service requirements;
the determining module 702 is specifically configured to determine a first coverage rate of the change code when the highest-level to-be-selected test case is executed, where the highest-level to-be-selected test case is a test case with a highest priority level in the to-be-selected test cases;
the adding module 703 is specifically configured to calculate a use case repetition rate based on the first coverage rate and the base coverage rate; judging whether the repetition degree of the use case is smaller than a preset repetition degree or not; when the case repetition degree is smaller than the preset repetition degree, adding the highest-level to-be-selected test case to the case library; when the repetition degree of the case is not less than the preset repetition degree, modifying the highest-level to-be-selected test case; returning to the step of determining the first coverage rate of the changed codes when the highest-level to-be-selected test case is executed;
the determining module 702 is specifically configured to determine, for each test case in other test cases to be selected, a second coverage rate of the change code when the test case is executed according to a sequence from high to low of a priority level; the other test cases to be selected comprise test cases except the highest-level test case to be selected in the test cases to be selected;
The adding module 703 is specifically configured to determine whether the second coverage rate is greater than a coverage rate before the test case is executed; when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library; otherwise, the test case is not added to the case library.
Optionally, the adding module 703 is specifically configured to determine, for each test case in the highest-level to-be-selected test cases in turn, a current coverage rate of the change code when the test case is executed; when the current coverage rate is the same as the coverage rate before the test case is executed, adding 1 to the repetition number of the case until all the test cases in the highest-level to-be-selected test case are executed, and obtaining the final repetition number of the case; and taking the ratio of the final use case repetition number to the total number of use cases as the use case repetition degree, wherein the total number of use cases is the total number of the highest-level to-be-selected test use cases.
Optionally, the acquiring module 701 is specifically configured to acquire a code to be tested; searching a starting node and an ending node in the code to be detected; obtaining a change file according to the starting node and the ending node; analyzing and obtaining a code implementation method corresponding to the change file through a source code analysis tool, and calculating a method parameter value corresponding to the change file; if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, taking the method realized by the code corresponding to the change file as a change method;
The determining module 702 is specifically configured to parse the change method through a coverage analysis tool, and generate a coverage report, where the coverage report includes an incremental coverage of the test case to be selected for the change method.
The test case screening device provided by the embodiment of the invention is a device applying the test case screening method, so that all the embodiments of the test case screening method are applicable to the device and can achieve the same or similar beneficial effects.
The embodiment of the invention also provides test case screening equipment, as shown in fig. 8, which comprises a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802 and the memory 803 complete communication with each other through the communication bus 804.
A memory 803 for storing a computer program;
the processor 801 is configured to implement the above-described method steps of test case screening when executing the program stored in the memory 803.
The communication bus mentioned by the test case screening device may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the test case screening device and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present invention, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the method steps of test case screening described above.
In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the method steps of test case screening described above is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus, device, computer-readable storage medium, and computer program product embodiments, the description is relatively simple, as relevant to the description of the method embodiments in part, since it is substantially similar to the method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (5)
1. A test case screening method, comprising:
acquiring a change code; the change code includes a code updated with respect to a reference code;
obtaining a test case to be selected; the test cases to be selected are the test cases to be added to the case library, and the test cases are used for testing codes;
determining the increment coverage rate of the test case to be selected for the change code, wherein the increment coverage rate represents the coverage rate of the change code when the test case to be selected is executed;
if the increment coverage rate meets the preset condition, adding the test case to be selected to a case library; otherwise, the test case to be selected is not added to the case library;
the test cases to be selected comprise test cases to be selected with different priority levels, wherein the priority levels are determined according to service requirements;
the determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
Determining a first coverage rate of a change code when executing the highest-level to-be-selected test case, wherein the highest-level to-be-selected test case is the test case with the highest priority level in the to-be-selected test cases;
if the increment coverage rate meets a preset condition, adding the test case to be selected to a case library; otherwise, not adding the test case to be selected to the case library, including:
calculating a case repetition degree based on the first coverage rate and the basic coverage rate;
judging whether the repetition degree of the use case is smaller than a preset repetition degree or not;
when the case repetition degree is smaller than the preset repetition degree, adding the highest-level to-be-selected test case to the case library;
when the repetition degree of the case is not less than the preset repetition degree, modifying the highest-level to-be-selected test case; returning to the step of determining the first coverage rate of the changed codes when the highest-level to-be-selected test case is executed;
the determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
determining a second coverage rate of the change codes when executing the test cases according to the order of the priority level from high to low for each of other test cases to be selected; the other test cases to be selected comprise test cases except the highest-level test case to be selected in the test cases to be selected;
And if the increment coverage meets a preset condition, adding the test case to be selected to a case library, wherein the method comprises the following steps:
judging whether the second coverage rate is larger than the coverage rate before executing the test case;
when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library; otherwise, the test case is not added to the case library.
2. The test case screening method according to claim 1, wherein the calculating the case repetition based on the first coverage and the base coverage includes:
determining the current coverage rate of the change code when executing the test case according to each test case in the highest-level to-be-selected test cases in sequence;
when the current coverage rate is the same as the coverage rate before the test case is executed, adding 1 to the repetition number of the case until all the test cases in the highest-level to-be-selected test case are executed, and obtaining the final repetition number of the case;
and taking the ratio of the final use case repetition number to the total number of use cases as the use case repetition degree, wherein the total number of use cases is the total number of the highest-level to-be-selected test use cases.
3. The test case screening method according to claim 1 or 2, wherein the acquiring the change code includes:
acquiring a code to be tested;
searching a starting node and an ending node in the code to be detected;
obtaining a change file according to the starting node and the ending node;
analyzing and obtaining a code implementation method corresponding to the change file through a source code analysis tool, and calculating a method parameter value corresponding to the change file;
if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, taking the method realized by the code corresponding to the change file as a change method;
the determining the increment coverage rate of the test case to be selected for the change code comprises the following steps:
analyzing the change method through a coverage rate analysis tool to generate a coverage rate report, wherein the coverage rate report comprises the increment coverage rate of the test case to be selected for the change method.
4. A test case screening device, comprising:
the acquisition module is used for acquiring the change code; the change code includes a code updated with respect to a reference code; obtaining a test case to be selected; the test cases to be selected are the test cases to be added to the case library, and the test cases are used for testing codes;
The determining module is used for determining the increment coverage rate of the test case to be selected for the change code, wherein the increment coverage rate represents the coverage rate of the test case to be selected for the change code when the test case to be selected is executed;
the adding module is used for adding the to-be-selected test cases to a case library if the increment coverage rate meets a preset condition; otherwise, the test case to be selected is not added to the case library;
the test cases to be selected comprise test cases to be selected with different priority levels, wherein the priority levels are determined according to service requirements;
the determining module is specifically configured to determine a first coverage rate of the change code when executing a highest-level to-be-selected test case, where the highest-level to-be-selected test case is a test case with a highest priority level in the to-be-selected test cases;
the adding module is specifically configured to calculate a use case repetition rate based on the first coverage rate and the basic coverage rate; judging whether the repetition degree of the use case is smaller than a preset repetition degree or not; when the case repetition degree is smaller than the preset repetition degree, adding the highest-level to-be-selected test case to the case library; when the repetition degree of the case is not less than the preset repetition degree, modifying the highest-level to-be-selected test case; returning to the step of determining the first coverage rate of the changed codes when the highest-level to-be-selected test case is executed;
The determining module is specifically configured to determine, for each test case in other test cases to be selected, a second coverage rate of the change code when the test case is executed according to a sequence from high priority to low priority; the other test cases to be selected comprise test cases except the highest-level test case to be selected in the test cases to be selected;
the adding module is specifically configured to determine whether the second coverage rate is greater than a coverage rate before the test case is executed; when the second coverage rate is larger than the coverage rate before the test case is executed, adding the test case to the case library; otherwise, the test case is not added to the case library.
5. The device according to claim 4, wherein the obtaining module is specifically configured to obtain a code to be tested; searching a starting node and an ending node in the code to be detected; obtaining a change file according to the starting node and the ending node; analyzing and obtaining a code implementation method corresponding to the change file through a source code analysis tool, and calculating a method parameter value corresponding to the change file; if the method parameter value corresponding to the change file is different from the method parameter value corresponding to the code to be tested, taking the method realized by the code corresponding to the change file as a change method;
The determining module is specifically configured to parse the change method through a coverage rate analysis tool, and generate a coverage rate report, where the coverage rate report includes an incremental coverage rate of the test case to be selected for the change method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296587.5A CN112783800B (en) | 2021-03-19 | 2021-03-19 | Test case screening method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296587.5A CN112783800B (en) | 2021-03-19 | 2021-03-19 | Test case screening method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112783800A CN112783800A (en) | 2021-05-11 |
| CN112783800B true CN112783800B (en) | 2023-09-05 |
Family
ID=75762791
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110296587.5A Active CN112783800B (en) | 2021-03-19 | 2021-03-19 | Test case screening method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783800B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115905031B (en) * | 2023-03-14 | 2023-06-23 | 云筑信息科技(成都)有限公司 | Test case recommendation method based on accurate quality assurance system |
| CN117435514B (en) * | 2023-12-21 | 2024-03-01 | 杭州罗莱迪思科技股份有限公司 | Extraction method for smoke emission test case |
| CN117667750B (en) * | 2024-01-31 | 2024-05-14 | 北京纷扬科技有限责任公司 | Test case generation method, test device, computer equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102005020899A1 (en) * | 2005-05-04 | 2006-11-16 | Siemens Ag | Method for preparing test coverage analysis involves testing of a program with several threads of error, which results e.g. from lack of synchronization between the threads with an conflicting access on common resources |
| CN106557413A (en) * | 2015-09-25 | 2017-04-05 | 伊姆西公司 | Based on the method and apparatus that code coverage obtains test case |
| CN107391376A (en) * | 2017-07-26 | 2017-11-24 | 成都科来软件有限公司 | A kind of regression testing method |
| CN107515826A (en) * | 2017-08-28 | 2017-12-26 | 广州阿里巴巴文学信息技术有限公司 | Test case accurate recommendation method, device, system, equipment and storage medium |
| US10146668B1 (en) * | 2013-12-20 | 2018-12-04 | EMC IP Holding Company LLC | Modeling code coverage in software life cycle |
| CN110471829A (en) * | 2018-05-09 | 2019-11-19 | 阿里巴巴集团控股有限公司 | The inspection method and device of software code test coverage |
| CN110502447A (en) * | 2019-08-30 | 2019-11-26 | 西安邮电大学 | A kind of regression test case priority ordering method based on figure |
| CN110532174A (en) * | 2019-07-24 | 2019-12-03 | 平安科技（深圳）有限公司 | Calculate method, apparatus, computer equipment and the storage medium of incremental code coverage rate |
| CN111506505A (en) * | 2020-04-13 | 2020-08-07 | 北京奇艺世纪科技有限公司 | Test case obtaining method and device |
| CN111858292A (en) * | 2019-09-12 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Screening method, screening system, computer device and storage medium of test case |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030093716A1 (en) * | 2001-11-13 | 2003-05-15 | International Business Machines Corporation | Method and apparatus for collecting persistent coverage data across software versions |
| US8381184B2 (en) * | 2008-09-02 | 2013-02-19 | International Business Machines Corporation | Dynamic test coverage |
-
2021
- 2021-03-19 CN CN202110296587.5A patent/CN112783800B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102005020899A1 (en) * | 2005-05-04 | 2006-11-16 | Siemens Ag | Method for preparing test coverage analysis involves testing of a program with several threads of error, which results e.g. from lack of synchronization between the threads with an conflicting access on common resources |
| US10146668B1 (en) * | 2013-12-20 | 2018-12-04 | EMC IP Holding Company LLC | Modeling code coverage in software life cycle |
| CN106557413A (en) * | 2015-09-25 | 2017-04-05 | 伊姆西公司 | Based on the method and apparatus that code coverage obtains test case |
| CN107391376A (en) * | 2017-07-26 | 2017-11-24 | 成都科来软件有限公司 | A kind of regression testing method |
| CN107515826A (en) * | 2017-08-28 | 2017-12-26 | 广州阿里巴巴文学信息技术有限公司 | Test case accurate recommendation method, device, system, equipment and storage medium |
| CN110471829A (en) * | 2018-05-09 | 2019-11-19 | 阿里巴巴集团控股有限公司 | The inspection method and device of software code test coverage |
| CN110532174A (en) * | 2019-07-24 | 2019-12-03 | 平安科技（深圳）有限公司 | Calculate method, apparatus, computer equipment and the storage medium of incremental code coverage rate |
| CN110502447A (en) * | 2019-08-30 | 2019-11-26 | 西安邮电大学 | A kind of regression test case priority ordering method based on figure |
| CN111858292A (en) * | 2019-09-12 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Screening method, screening system, computer device and storage medium of test case |
| CN111506505A (en) * | 2020-04-13 | 2020-08-07 | 北京奇艺世纪科技有限公司 | Test case obtaining method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112783800A (en) | 2021-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112783800B (en) | Test case screening method and device | |
| US9904614B2 (en) | Source code inspection and verification | |
| US9715441B2 (en) | Risk-based test coverage and prioritization | |
| US8117598B2 (en) | Method and apparatus to automatically identify specific code changes to probabilistically exclude from regression | |
| CN111488578A (en) | Continuous vulnerability management for modern applications | |
| US8161458B2 (en) | Method and apparatus to increase efficiency of automatic regression in “two dimensions” | |
| US9880832B2 (en) | Software patch evaluator | |
| Weigert et al. | Practical experiences in using model-driven engineering to develop trustworthy computing systems | |
| CN110879781B (en) | Program debugging method, device, electronic equipment and computer readable storage medium | |
| CN105550325A (en) | Data management method and device | |
| US20200097579A1 (en) | Detecting anomalous transactions in computer log files | |
| US10824541B1 (en) | System and method for test data fabrication | |
| CN111858369A (en) | Memory monitoring method, device, equipment and storage medium | |
| CN111435327B (en) | Log record processing method, device and system | |
| CN116578282A (en) | Code generation method, device, electronic equipment and medium | |
| CN115129575A (en) | Code coverage result generation method and device | |
| CN112416417A (en) | Code amount statistical method and device, electronic equipment and storage medium | |
| CN113254325A (en) | Test case processing method and device | |
| CN117130945B (en) | Test method and device | |
| CN111104318B (en) | Unit test result export method and device and electronic equipment | |
| CN112035119B (en) | Data deleting method and device | |
| Schuckert et al. | Diffcult SQLi Code Patterns for Static Code Analysis Tools | |
| CN113010431B (en) | Java unit test case generation method and device | |
| US20240054232A1 (en) | Build process security framework | |
| Davies et al. | Usage of simulation for QA in real-time network environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |