CN110471829B

CN110471829B - Method and device for checking software code test coverage rate

Info

Publication number: CN110471829B
Application number: CN201810440103.8A
Authority: CN
Inventors: 石晓海; 秦承刚; 张永超
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2018-05-09
Filing date: 2018-05-09
Publication date: 2023-07-18
Anticipated expiration: 2038-05-09
Also published as: CN110471829A

Abstract

The application provides a method for checking software code test coverage rate, which comprises the following steps: testing the software codes containing the patch code segments according to the test set; determining the tested coverage rate of the patch code section, wherein the tested coverage rate of the patch code section is used for measuring the code quantity of the patch code section executed in the test process; and determining a coverage rate test result of the patch code section according to the coverage rate of the patch code section to be tested. The coverage rate checking process of each submitted code is integrated with the continuous testing process, so that the coverage rate checking is performed through the continuous submitting process of the code, and the continuous coverage rate checking is formed. And if the coverage rate of the code submitted at this time does not meet the requirement, the test is performed again, so that the coverage check of the software patch is more timely and targeted.

Description

Method and device for checking software code test coverage rate

Technical Field

The application relates to the field of software testing, in particular to a method and a device for checking software code testing coverage rate, and further relates to electronic equipment and a computer readable storage medium.

Background

In CI (Continuous integration, continuously integrated) systems, software quality is generally detected and guaranteed through testing. At the same time, the coverage of the code under test may be checked for evaluating the reliability of the software test.

The prior art inspection of the coverage of the code being tested is performed as follows: in the process of continuous integration of software, software developers usually construct executable software versions after submitting codes, and test the executable software versions. After a test fails, the developer will revise and submit the test again until the test passes. After all test items have passed, a new version is released, if appropriate. There may also be a total coverage statistic to know the proportion of the whole software code that is covered in the test. For example, a piece of software has 1 ten thousand lines of code, which 1 ten thousand lines may be tested for 5000 lines, and the coverage of all the 1 ten thousand lines of code for the entire piece of software is 50%.

In this case, the coverage check is a check at the time of software testing for the coverage at which all the codes constituting the entire software of the executable version are tested. When using this inspection method, the code amount of one-time inspection is very large, and the inspection granularity is not fine enough. Because it is difficult to judge the coverage rate of the tested part of the code submitted each time, problems cannot be found timely in the large-scale software development process, and whether the code submitted each time meets the quality requirement cannot be guaranteed.

Disclosure of Invention

The application provides a method for checking the test coverage rate of software codes, which aims to solve the technical problems that the code quantity of one-time check is very large and the check granularity is not fine in the coverage rate check in the prior art.

The application also provides a device for checking the software code test coverage rate, and also relates to electronic equipment and a computer readable storage medium.

The method for checking the software code test coverage rate comprises the following steps:

testing the software codes containing the patch code segments according to the test set;

determining the tested coverage rate of the patch code section, wherein the tested coverage rate of the patch code section is used for measuring the code quantity of the patch code section executed in the test process;

and determining a coverage rate test result of the patch code section according to the coverage rate of the patch code section to be tested.

Optionally, the determining the coverage rate of the patch code section to be tested includes:

acquiring execution information of the patch code section;

and determining the tested coverage rate of the patch code section according to the execution information.

Optionally, the execution information includes at least one of:

line number of the code executed in the patch code section;

The number of lines of code executed in the patch code section;

the number of times each line of code in the patch code section is executed.

Optionally, the determining the coverage rate of the patch code section to be tested according to the execution information includes:

determining the number of lines of the codes executed in the patch code section according to the line numbers of the codes executed in the patch code section, and taking the ratio of the number of lines of the codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section; or,

determining the number of lines of the codes executed in the patch code section, and taking the ratio of the number of lines of the codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section; or,

and determining the number of lines of codes executed in the patch code section according to the number of times each line of codes is executed in the patch code section, and taking the ratio of the number of lines of codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section.

Optionally, the determining the coverage rate test result of the patch code section according to the coverage rate tested by the patch code section includes:

if the tested coverage rate of the patch code section reaches a code section coverage rate threshold value, determining that the patch code section passes a coverage rate test;

and if the tested coverage rate of the patch code section does not reach the code section coverage rate threshold value, determining that the patch code section does not pass the coverage rate test.

Optionally, if the patch code section fails the coverage test, the method further comprises:

the software code containing the patch code segments is tested again.

Optionally, the testing the software code including the patch code section again includes:

generating a supplementary test set aiming at codes which are not tested in the patch code segment;

updating the supplemental test set into the test set;

and testing the software codes containing the patch code segments again according to the updated test set.

Optionally, the generating a supplemental test set for the code that is not tested in the patch code section includes:

determining the identification of an uploader corresponding to the patch code section according to the identification of the patch code section;

And according to the identification of the uploading user, notifying the uploading user to generate a supplementary test set aiming at the codes which are not covered by the test in the supplementary code segment.

Optionally, if, after testing the software code containing the patch code section again, the coverage rate of the patch code section tested reaches the coverage rate threshold of the code section, the method further includes:

calculating the overall coverage rate of the software code containing the patch code section;

and if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold, acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated, wherein the first operation parameter is used for representing the operation quality of the software code containing the patch code section when the software code containing the patch code section is tested and operated.

Optionally, the method further comprises:

and if the first operation parameter reaches a first operation index threshold value, determining that the software code containing the patch code section passes an operation quality test.

Optionally, if the patch code section passes the coverage test, the method further comprises:

and acquiring a second operation parameter generated when the patch code section is tested and operated, wherein the second operation parameter is used for representing the operation quality of the patch code section when the patch code section is tested and operated.

Optionally, the method further comprises:

and if the second operation parameter does not reach the second operation index threshold value, sending the second operation parameter to an uploader of the patch code section, and informing the uploader of optimizing the patch code section.

Optionally, the method further comprises:

integrating the obtained optimized patch code section into the software, and then testing again;

acquiring a third operation parameter of the optimized patch code section when tested and operated;

and if the third operation parameter reaches the third operation index threshold value, determining that the optimized patch code section passes an operation quality test.

The application also provides a method for checking the coverage rate of the patch code section in the software continuous integration scene, which comprises the following steps:

detecting whether a patch code section is integrated into current software;

when detecting that the patch code segments are integrated into the current software, testing the software codes containing the patch code segments according to a test set;

determining the tested coverage rate of the patch code section;

and determining the patch code sections with the tested coverage rate reaching a first threshold as patch code sections meeting the integration condition.

The device for checking the software code test coverage rate comprises:

the first test unit is used for testing the software codes containing the patch code segments according to the test set;

the coverage rate determining unit is used for determining the tested coverage rate of the patch code section, and the tested coverage rate of the patch code section is used for measuring the code quantity of the patch code section executed in the test process;

and the coverage rate test result determining unit is used for determining the coverage rate test result of the patch code section according to the tested coverage rate of the patch code section.

Optionally, the coverage determining unit includes:

an execution information acquisition subunit, configured to acquire execution information of the patch code section;

and the coverage rate acquisition subunit is used for determining the tested coverage rate of the patch code section according to the execution information.

Optionally, the execution information obtaining subunit is configured to obtain at least one of the following execution information:

the line number of the code executed in the patch code section, and the number of times each line of the code is executed in the patch code section.

Optionally, the coverage rate obtaining subunit includes:

The coverage rate obtaining second-level first subunit is configured to determine, according to the line number of the code executed in the patch code section, and use the ratio of the line number of the code executed in the patch code section to the line number of all codes in the patch code section as the coverage rate of the patch code section to be tested; or,

the coverage rate obtaining second-level subunit is used for determining the number of lines of the codes executed in the patch code section, and taking the ratio of the number of lines of the codes executed in the patch code section to the number of lines of all codes in the patch code section as the coverage rate of the patch code section to be tested; or,

the coverage rate obtaining second-level third subunit is configured to determine, according to the number of times that each line of code in the patch code section is executed, the number of lines of the code executed in the patch code section, and use, as the coverage rate of the patch code section to be tested, the ratio of the number of lines of the code executed in the patch code section to the number of lines of all codes in the patch code section.

Optionally, the coverage test result determining unit includes:

The coverage rate test result determines a first subunit, which is used for determining that the patch code section passes the coverage rate test if the tested coverage rate of the patch code section reaches a code section coverage rate threshold;

and the coverage rate test result determines a second subunit, which is used for determining that the patch code section fails the coverage rate test if the tested coverage rate of the patch code section does not reach the code section coverage rate threshold value.

Optionally, the apparatus further includes:

and the second test unit is used for testing the software code containing the patch code segment again.

Optionally, the second test unit includes:

a supplemental test set generation subunit, configured to generate a supplemental test set for a code that is not tested in the patch code segment;

a supplemental test set update subunit configured to update the supplemental test set into the test set;

and the second testing subunit is used for testing the software codes containing the patch code segments again according to the updated testing set.

Optionally, the supplemental test set generation subunit includes:

the uploading user identification determining secondary subunit is used for determining the identification of the uploading user corresponding to the patch code section according to the identification of the patch code section;

And the notification second-level subunit is used for notifying the uploading user to generate a supplementary test set for the codes which are not covered by the test in the supplementary code segment according to the identification of the uploading user.

Optionally, the apparatus further includes:

the whole coverage rate calculation unit is used for calculating the whole coverage rate of the software code containing the patch code section;

and the first operation parameter acquisition unit is used for acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold value, wherein the first operation parameter is used for representing the operation quality of the software code containing the patch code section when the software code containing the patch code section is tested and operated.

Optionally, the apparatus further includes:

and the first operation quality determining unit is used for determining that the software code containing the patch code section passes the operation quality test if the first operation parameter reaches a first operation index threshold value.

Optionally, the apparatus further includes:

the second operation parameter acquisition unit is used for acquiring a second operation parameter generated when the patch code section is tested and used for representing the operation quality of the patch code section when the patch code section is tested and operated.

Optionally, the apparatus further includes:

and the notification optimizing unit is used for sending the second operation parameter to an uploader of the patch code section if the second operation parameter does not reach a second operation index threshold value, and notifying the uploader to optimize the patch code section.

Optionally, the apparatus further includes:

the third testing unit is used for integrating the obtained optimized patch code section into the software and then testing again;

a third operation parameter obtaining unit, configured to obtain a third operation parameter when the optimized patch code section is tested and operated;

and the second operation quality determining unit is used for determining that the optimized patch code section passes an operation quality test if the third operation parameter reaches the third operation index threshold value.

The electronic equipment that this application provided includes:

a memory and a processor;

the memory is for storing computer-executable instructions, and the processor is for executing the computer-executable instructions:

The present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the following operations:

According to the technical scheme, the coverage rate checking process is incorporated as part of the software continuous integration process, so that the coverage rate of the code to be tested submitted each time is checked, and therefore, how the coverage rate of the code to be tested submitted each time is exactly can be known. And if the coverage rate of the code submitted at this time does not meet the requirement, the test is performed again, so that the coverage check of the software patch is more timely and targeted.

Drawings

FIG. 1 is a process flow diagram of an embodiment of a method for checking software code test coverage provided herein;

Fig. 2 is an application scenario schematic diagram of a method for checking software code test coverage provided in the present application;

FIG. 3 is a schematic diagram of an embodiment of an inspection apparatus for software code test coverage provided herein;

fig. 4 is a schematic diagram of an embodiment of an electronic device provided herein.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.

Fig. 1 is a process flow chart of an embodiment of a method for checking the coverage of software code test. The technical scheme of the method for checking the software code test coverage rate provided in the present application is described below with reference to a process flow chart of an embodiment of the method for checking the software code test coverage rate shown in fig. 1.

The process flow of the embodiment of the software code test coverage checking method shown in fig. 1 includes:

Step S101, testing the software codes containing the patch code segments according to the test set.

This step S101 aims at testing the software code containing the patch code section in preparation for determining the condition of the software code containing the patch code section or the patch code section according to the test result in the subsequent step.

Wherein, the patch is called patch in the software field, and Chinese can be called patch. The patch refers to new codes added and integrated by the current software again when the current software is used for solving the defects of the current software or enhancing one function. Correspondingly, the patch code section refers to the code included in the patch. For example, there are 1 ten thousand lines of current software, but the current software lacks a certain function. In this case, a piece of code for realizing the function may be designed, for example, 30 lines of code for realizing the function. The 30 lines of code implementing this function may be integrated into the current software, with the 30 lines of complementary code corresponding to the patches of the current software.

The test refers to a process of executing the software code including the patch code section in order to verify the operation condition of all the codes included in the whole software or in order to verify the operation condition of a part of the codes of the software. Because the test process in the technical scheme of the application is mainly performed on whether the code is run or not, the test process is mainly performed on the function of the code. The test may be aided by a test tool. Testing software is a well-established prior art in the field of computer software, and can be accomplished by one of ordinary skill in the art and will not be described in detail herein.

The test set refers to a set of test data in a functional test set used in testing a software function, the functional test set being prepared for a patch to be subjected to a functional test. Compared with the traditional test set required to be aimed at the whole software in the whole coverage rate inspection process of the software, the functional test set aiming at the patch in the technical scheme of the application is smaller, so that the patch can be subjected to functional test in a targeted manner during test, and the test efficiency is improved. For example, the test data may be a number of user identifications, user behavior data, etc., that are pre-generated or prepared. Typically, the test data may be generated by a test tool. When the software is tested by using the testing tool, the software testing tool enables the software to be tested to run according to the testing data, so that various running indexes of the software are judged.

Step S102, determining the tested coverage rate of the patch code section, wherein the tested coverage rate of the patch code section is used for measuring the code quantity of the patch code section executed in the test process.

This step S102 aims at determining the coverage of the patch code section under test to evaluate the quality, trustworthiness of the test according to the coverage of the patch code section under test. Since the quality of the running of the part of the code that is not tested cannot be determined if only a part of the code is run at the time of testing, although it may be that the test is run well, the reliability of the test results obtained in this case is not particularly high.

Optionally, the step S102 includes:

a) And acquiring the execution information of the patch code section.

Since step S101 is performed on the software code including the patch code section according to the test set, a compiling tool, such as a compiling tool gcc (GNU Compiler Collection, compiler kit), is used. After compiling the software code containing the patch code segment by using a compiling tool, executing information of each row of code recorded with the software containing the patch code segment is generated in the tested process.

In sub-step a) of the step S102, optionally, the execution information includes at least one of: the line number of the code executed in the patch code section, and the number of times each line of the code is executed in the patch code section.

When testing software code that contains patch code segments, typically not all code is run through, and possibly some code is not. The patch code section may not have all code running therein, and it is possible that one portion of the patch code section is running and another portion is not. The line number of the code executed in the patch code section is the line number of the code executed by the patch code section in test. The number of lines of the code executed in the patch code section refers to the total number of lines executed in the patch code section. The number of times each line of code in the patch code section is executed refers to the number of times each line of code in the patch code section is executed, for example, 1 time for a line, 0 time for a line, and so on.

B) And determining the tested coverage rate of the patch code section according to the execution information.

Substep B) of the present S102 is directed to determining the tested coverage of the patch code section according to the execution information, so as to evaluate the quality and credibility of the test according to the tested coverage of the patch code section. The tested coverage rate of the patch code section refers to the ratio of the number of lines running in the patch code section during testing to the total number of lines of the code of the patch code section.

Optionally, the substep B) of the step S102 includes:

the execution information in sub-step a) based on S102 includes the above three categories as described above. Accordingly, the sub-step B) of the step S102 may be implemented in the following three ways.

Mode one: determining the number of lines of the codes executed in the patch code section according to the line numbers of the codes executed in the patch code section, and taking the ratio of the number of lines of the codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section;

if the execution information in substep a) of S102 includes the line number of the code being executed in the patch code section, this step B) may be implemented in a manner. For example, the number of line numbers of the code executed in the patch code section may be counted to determine the number of line numbers of the code executed in the patch code section. And then the ratio of the number of lines of the executed codes in the patch code section to the number of lines of all codes in the patch code section can be used as the tested coverage rate of the patch code section.

Mode two: determining the number of lines of the codes executed in the patch code section, and taking the ratio of the number of lines of the codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section.

If the execution information in sub-step a) of S102 includes the number of lines of code executed in the patch code section, this step B) may be implemented in a second way. For example, the ratio of the number of lines of the code executed in the patch code section to the number of lines of all the codes in the patch code section is calculated, and the calculated ratio result is determined as the tested coverage rate of the patch code section.

Mode three: and determining the number of lines of codes executed in the patch code section according to the number of times each line of codes is executed in the patch code section, and taking the ratio of the number of lines of codes executed in the patch code section to the number of lines of all codes in the patch code section as the tested coverage rate of the patch code section.

If the execution information in substep a) of S102 includes the number of times each line of code in the patch code section is executed, this step B) may be implemented in a third way. For example, if the number of times that the code of the line in the patch code segment is executed is 0, the number of times that the code of the line is executed is 1, the number of times that the code of the line is executed is 2, etc., the number of lines that all the lines executed with the number of times equal to or greater than 1 is counted, the number of lines that the code is executed in the patch code segment can be determined. And further, the ratio of the number of lines of the executed codes in the patch code section to the number of lines of all codes in the patch code section is used as the tested coverage rate of the patch code section.

And step S103, determining a coverage rate test result of the patch code section according to the coverage rate tested by the patch code section.

This step S103 is intended to determine a coverage test result of the patch code section according to the coverage of the patch code section to be tested, where the coverage test result of the patch code section means that the patch code section passes the coverage test.

Optionally, the step S103 includes:

a) And if the tested coverage rate of the patch code section reaches a code section coverage rate threshold value, determining that the patch code section passes the coverage rate test. For example, the coverage rate of the tested patch code section is 93%, and if the coverage rate threshold of the code section is 90%, the patch code section is determined to pass the coverage rate test.

B) And if the tested coverage rate of the patch code section does not reach the code section coverage rate threshold value, determining that the patch code section does not pass the coverage rate test. For example, the coverage rate of the patch code section to be tested is 60%, and if the coverage rate threshold of the code section is not 90%, it is determined that the patch code section fails the coverage rate test.

Optionally, after the sub-step B) of the step S103 is performed, that is, if the coverage rate of the patch code section tested does not reach the code section coverage rate threshold, it is determined that the patch code section fails the coverage rate test, the method further includes:

Step S104, the software code containing the patch code section is tested again.

This step S104 aims to retest the software code containing the patch code section, so as to reconfirm the tested coverage rate of the patch code section through retesting, and further evaluate the quality and credibility of the test according to the tested coverage rate of the patch code section.

Optionally, the step S104 includes:

the first step: and generating a supplementary test set for codes which are not tested in the patch code segment.

The supplemental test set refers to a test set generated for code in the patch code section that is not tested. The supplemental test set is intended to facilitate being run-time covered when code in the patch code segment that is not tested is tested again. Due to insufficient test data in the test set, there may be code in the patch code segment that is not tested. Therefore, a supplemental test set is generated based on the code in the patch code segment that is not tested, for causing the code in the patch code segment that is not tested to be executed when retested by the generated supplemental test set.

Optionally, the first step includes:

1) And determining the identification of the uploader corresponding to the patch code section according to the identification of the patch code section. The identification of the patch code section is the code number of the patch code section. For example, the identification of the patch code section may be a field code101. The identification of the uploader corresponding to the patch code section is used to characterize that the patch code section is uploaded by the corresponding developer. For example, the identification of the uploader to which the patch code section corresponds may be user101. The determining, according to the identification of the patch code section, the identification of the uploader corresponding to the patch code section may be implemented in the following manner: and determining the identification of the uploader corresponding to the patch code section according to the mapping relation between the identification of the patch code section and the identification of the uploader corresponding to the patch code section. Optionally, the mapping relationship between the identification of the patch code section and the identification of the uploader corresponding to the patch code section may be characterized by looking up a mapping table.

2) And according to the identification of the uploading user, notifying the uploading user to generate a supplementary test set aiming at the codes which are not covered by the test in the supplementary code segment.

After executing step 1) to determine the identification of the uploader corresponding to the patch code section according to the identification of the patch code section, this step aims at notifying the uploader to generate a supplemental test set for the code which is not covered by the test in the supplemental code section.

By the uploader generating the supplemental test set for code in the supplemental code segment that is not covered by the test, the pertinence of generating the supplemental test set may be enhanced. Since the uploader to which the patch code section corresponds is most aware of the functionality of the patch code section, the supplemental test set generated by the uploader for the code in the supplemental code section that is not covered by the test has a stronger correspondence with the code in the supplemental code section that is not covered by the test, so that the code in the patch code section that is not tested can be executed when retested by means of the generated supplemental test set.

And a second step of: updating the supplemental test set into the test set.

This step aims at updating the supplemental test set to the test set. When integrating the patch code sections into the current software, the supplemental test set also needs to be updated into the test set in order to meet the coverage requirements when the supplemental code sections are tested accordingly. When integrating the patch code sections into the current software, if the test set lacks test data corresponding to the patch code sections, it may result in lower coverage of the supplemental code sections being tested. Thus, the supplemental test set is updated into the test set so that code in the patch code segment that is not tested can be executed when retested by means of the generated supplemental test set.

And a third step of: and testing the software codes containing the patch code segments again according to the updated test set.

The step aims at retesting by means of the generated supplementary test set, so that codes which are not tested in the patch code segments can be executed when retesting is carried out again, the coverage rate requirement of the tested patch code segments is met, and the quality and the credibility of the test are further evaluated according to the coverage rate of the tested patch code segments. Specific testing procedures are available in the art and are not described in detail herein.

Optionally, after the software code including the patch code section is tested again in the step S104 a), if the coverage rate of the patch code section tested reaches the code section coverage rate threshold, the method further includes:

step S105, calculating the overall coverage of the software code including the patch code section.

Based on the description of the coverage rate of the patch code section to be tested, accordingly, it is known that the overall coverage rate of the software code containing the patch code section refers to the coverage rate of all the codes when the software code containing the patch code section is tested. The manner in which the overall coverage of the software code containing patch code sections is calculated may be referred to above as obtaining the coverage of the patch code sections being tested, or may take other forms, and is not limited thereto.

Optionally, based on step S105, the method further includes:

and step S106, if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold, acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated, wherein the first operation parameter is used for representing the operation quality when the software code containing the patch code section is tested and operated.

The step S106 is aimed at obtaining a first operation parameter generated when the software code including the patch code segment is tested and operated under the condition that the overall coverage rate of the software code including the patch code segment reaches the overall coverage rate threshold, so as to obtain the operation quality of the software code including the patch code segment when the software code including the patch code segment is tested and operated according to the first operation parameter generated when the software code including the patch code segment is tested and operated.

The first operation parameter generated when the software code containing the patch code section is tested and operated can be parameters such as the utilization rate of a central processing unit, used memory, disk input and output rate, network throughput and the like for representing the operation quality when the software code containing the patch code section is tested and operated. The utilization rate of the central processing unit refers to the proportion of resources in the central processing unit occupied by the running program. The resources in the central processing unit include an operator unit, an arithmetic logic unit, a controller unit, a register unit, and the like. Used memory refers to the memory that is being occupied by a running program. The disk input/output rate refers to the number of disk read/write operations per unit time. Network throughput refers to the number of user requests processed per unit time.

Optionally, based on step S106, the method further includes:

step S107, if the first operation parameter reaches a first operation index threshold value, determining that the software code containing the patch code segment passes an operation quality test.

After obtaining the first operation parameter for characterizing the operation quality of the software code including the patch code section when the software code including the patch code section is tested based on step S106, this step S107 is aimed at determining that the software code including the patch code section has passed the operation quality test after determining that the first operation parameter reaches the first operation index threshold. Wherein, based on the following first parameters listed in step S106: the first operation index threshold may be a utilization index threshold, a used memory index threshold, a disk input/output rate index threshold, and a network throughput index threshold of the central processing unit.

And after determining that the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold through the step S106, and further determining that the software code containing the patch code section passes the operation quality test through the step S107, the software code containing the patch code section passing the operation quality test can be determined to obtain the assurance of the overall coverage rate, so that the test reliability is higher.

Optionally, based on sub-step B) of step S103, if the coverage of the patch code section tested does not reach the code section coverage threshold, after determining that the patch code section fails the coverage test, the method further includes:

step S108, obtaining a second operation parameter generated when the patch code section is tested, wherein the second operation parameter is used for representing the operation quality of the patch code section when the patch code section is tested.

The second operating parameter is used for representing the operating quality of the patch code section when the patch code section is tested to run. The second operation parameter may also be a parameter used for characterizing operation quality of the software code including the patch code section when the software code including the patch code section is tested, such as a utilization rate of a central processing unit corresponding to the patch code section when the patch code section is tested and running, a used memory, a disk input/output rate, and a network throughput. Since the corresponding server may be a certain server, the second parameters such as the utilization rate of the central processing unit, the used memory, the disk input/output rate, and the network throughput on the corresponding server for characterizing the running quality of the patch code section when tested can be obtained separately.

Optionally, based on step S108, the method further includes:

step S109, if the second operation parameter does not reach the second operation index threshold, sending the second operation parameter to an uploader of the patch code section, so as to notify the uploader to optimize the patch code section.

This step S109 is intended to determine that the second operation parameter for characterizing the operation quality of the patch code section when the tested operation has not reached a second operation index threshold, send the second operation parameter to an uploader of the patch code section, and notify the uploader to optimize the patch code section.

The step of the uploader optimizing the patch code section means that the uploader optimizes the corresponding amplitude according to the second operation parameter which does not reach a second operation index threshold. For example, if the second operating parameter differs significantly from the second operating index threshold, a greater degree of optimization is performed. And if the second operation parameter is smaller than the second operation index threshold value, performing smaller-amplitude optimization. By sending the second operating parameter to the uploader of the patch code section to inform the uploader of optimizing the patch code section, the magnitude of the optimization of the patch code section by the uploader can be more targeted and efficient.

After notifying the uploader to optimize the patch code section based on step S109, the method further includes:

step S110, after integrating the obtained optimized patch code segments into the software, testing again.

And step S111, obtaining a third operation parameter of the optimized patch code section when tested and operated.

And step S112, if the third operation parameter reaches the third operation index threshold value, determining that the optimized patch code section passes an operation quality test.

Step S110, step S111, and step S112 are aimed at integrating the obtained optimized patch code segment into the software and then testing again, and obtaining a third operation parameter when the optimized patch code segment is tested and operated, and further determining that the optimized patch code segment passes an operation quality test if the third operation parameter reaches the third operation index threshold, so as to ensure that the optimized patch code segment reaches an operation quality requirement.

In addition to the method for checking the coverage rate of the software code test, the application also provides a method for checking the coverage rate of the patch code section in the software continuous integration scene.

The method for checking the coverage rate of the patch code sections in the software continuous integration scene comprises the following steps:

step one, it is detected whether a patch code section is integrated into the current software.

And step two, when detecting that the patch code sections are integrated into the current software, testing the software codes containing the patch code sections according to the test set.

And thirdly, determining the tested coverage rate of the patch code section.

And step four, determining the patch code sections with the tested coverage rate reaching a first threshold as patch code sections meeting the integration condition.

Since the method of checking the coverage of patch code sections in the software continuous integration scenario is similar to the method of checking the test coverage of software code provided above, it will not be described in detail.

The technical effects obtained by the technical scheme of the application are analyzed by combining with an application scene schematic diagram of the method for checking the software code test coverage shown in fig. 2.

As shown in FIG. 2, where the developer is referred to as the writer of the patch code segment described in the previous embodiments, the code repository may include all the code of the current software, and the developer may integrate the patch code segment written by the developer into the code of the current software. The test system in fig. 2 may be the test tool referred to above, which may test software code in a code repository. Coverage checking refers to checking the coverage of the patch code section to be tested as described above. A tester refers to a person testing the software containing the patch code section. Test cases, i.e., test events, refer to the data in the test set described above that is used to test software code. If the test system fails to test the software containing the patch code section, for example, the coverage rate of the tested patch code section does not meet the requirement, or the bug appears in the software containing the patch code section, the process is re-executed until the test is successful, and the release of the version can be carried out.

According to the technical scheme, the coverage rate checking process is incorporated as part of the software continuous integration process, so that the coverage rate of the code to be tested submitted each time is checked, and therefore, how the coverage rate of the code to be tested submitted each time is exactly can be known. By integrating the coverage check process of each submitted code with the continuous test process, so that the check of coverage runs through to the continuous submission process of the code, a so-called continuous coverage check (Continuous Coverage Check) is formed. The method is beneficial to a specific developer to supplement test cases in time, the quality of control software is more targeted and more timely, and the quality of codes is effectively ensured. For example, a submitted patch at a time has 100 lines of code, the coverage tested can be counted for the 100 lines of code when testing, if only 60 lines of code are found to be covered by the test, the coverage of the 100 lines of patch is 60%, while knowing who submitted code is not covered by the test, or the coverage is insufficient.

Optionally, if the code submitted at a time is not covered by the test or coverage rate is insufficient, the system can consider that the system has potential quality problems and refuses integration, and a specific developer supplements the test case in time and optimizes the patch code segment in time, so that the quality of the control software is more targeted and timely. For example, if 60% coverage is deemed to be below the expected requirement, supplemental testing, code optimization may be performed until coverage reaches the expected requirement, and the operating parameters reach the operating index.

The application also provides a device for checking the software code test coverage rate. Fig. 3 is a schematic diagram of an embodiment of an inspection apparatus for software code test coverage provided in the present application. Since the apparatus embodiments are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the corresponding descriptions of the method embodiments provided above for relevant parts.

The device for checking the software code test coverage in the embodiment shown in fig. 3 includes:

a first test unit 301, configured to test a software code including a patch code section according to a test set;

a coverage rate determining unit 302, configured to determine a coverage rate of the patch code section, where the coverage rate of the patch code section is used to measure a code amount of the patch code section executed in a testing process;

and the coverage rate test result determining unit 303 is configured to determine a coverage rate test result of the patch code section according to the coverage rate of the patch code section to be tested.

Optionally, the coverage determining unit 302 includes:

Optionally, the coverage rate obtaining subunit includes:

Optionally, the coverage test result determining unit 303 includes:

Optionally, the apparatus further includes:

Optionally, the second test unit includes:

a supplementary test set generating subunit, configured to generate a supplementary test set according to the code that is not tested in the patch code segment;

Optionally, the supplemental test set generation subunit includes:

And the notification second-level subunit is used for notifying the uploader to generate a supplemental test set aiming at the codes which are not covered by the test in the supplemental code segment.

Optionally, the apparatus further includes:

The application also provides electronic equipment. Fig. 4 shows an embodiment of the electronic device. Since the embodiment of the electronic device is substantially similar to the method embodiment, the description is relatively simple, and the relevant portions are referred to the corresponding descriptions of the method embodiment provided above.

The electronic device in the embodiment shown in fig. 4 includes:

a memory 401 and a processor 402;

The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the following operations:

While the preferred embodiment has been described, it is not intended to limit the invention thereto, and any person skilled in the art may make variations and modifications without departing from the spirit and scope of the present invention, so that the scope of the present invention shall be defined by the claims of the present application.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims

1. A method for checking software code test coverage, comprising:

testing software codes containing patch code segments according to a test set, wherein the test set is used for testing patches to be subjected to functional test;

determining a coverage rate test result of the patch code section according to the tested coverage rate of the patch code section, and if the patch code section fails the coverage rate test, retesting the software code containing the patch code section, wherein the method comprises the following steps: generating a supplementary test set aiming at codes which are not tested in the patch code segment; updating the supplemental test set into the test set; testing the software codes containing the patch code segments again according to the updated test set;

If the coverage rate of the tested patch code section reaches the coverage rate threshold of the code section after the software code containing the patch code section is tested again, the method further comprises the following steps: calculating the overall coverage rate of the software code containing the patch code section; and if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold, acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated, wherein the first operation parameter is used for representing the operation quality of the software code containing the patch code section when the software code containing the patch code section is tested and operated.

2. The method of claim 1, wherein said determining coverage of the patch code section under test comprises:

acquiring execution information of the patch code section;

3. The method of claim 2, wherein the execution information includes at least one of:

line number of the code executed in the patch code section;

the number of lines of code executed in the patch code section;

The number of times each line of code in the patch code section is executed.

4. A method as defined in claim 3, wherein determining the coverage of the patch code section under test based on the execution information comprises:

5. The method of claim 1, wherein the determining a coverage test result for the patch code section based on the coverage of the patch code section being tested comprises:

6. The method of claim 1, wherein the generating a supplemental test set for code in the patch code section that is not tested comprises:

and according to the identification of the uploader, notifying the uploader to generate a supplementary test set aiming at the codes which are not covered by the test in the patch code segment.

7. The method according to claim 1, wherein the method further comprises:

8. The method of claim 5, wherein if the patch code section passes a coverage test, the method further comprises:

9. The method of claim 8, wherein the method further comprises:

10. The method according to claim 9, wherein the method further comprises:

and if the third operation parameter reaches a third operation index threshold value, determining that the optimized patch code section passes an operation quality test.

11. A method for checking coverage rate of patch code sections in a software continuous integration scene, comprising:

Detecting whether a patch code section is integrated into current software;

when detecting that the patch code segments are integrated into the current software, testing the software codes containing the patch code segments according to a test set, wherein the test set is used for testing patches to be subjected to functional test;

determining the tested coverage rate of the patch code section, and if the patch code section fails the coverage rate test, testing the software code containing the patch code section again, wherein the method comprises the following steps: generating a supplementary test set aiming at codes which are not tested in the patch code segment; updating the supplemental test set into the test set; testing the software codes containing the patch code segments again according to the updated test set;

determining the patch code sections with the tested coverage rate reaching a first threshold as patch code sections meeting the integration condition;

12. An inspection apparatus for software code test coverage, comprising:

the first test unit is used for testing the software codes containing the patch code segments according to a test set, wherein the test set is used for testing the patches to be subjected to the functional test;

the coverage rate test result determining unit is configured to determine a coverage rate test result of the patch code section according to the coverage rate of the patch code section to be tested, and if the patch code section fails the coverage rate test, retest the software code containing the patch code section, where the coverage rate test result determining unit includes: generating a supplementary test set aiming at codes which are not tested in the patch code segment; updating the supplemental test set into the test set; testing the software codes containing the patch code segments again according to the updated test set;

wherein if the coverage rate of the tested patch code section reaches the code section coverage rate threshold after the software code containing the patch code section is tested again, the device further comprises: the whole coverage rate calculation unit is used for calculating the whole coverage rate of the software code containing the patch code section; and the first operation parameter acquisition unit is used for acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold value, wherein the first operation parameter is used for representing the operation quality of the software code containing the patch code section when the software code containing the patch code section is tested and operated.

13. The apparatus according to claim 12, wherein the coverage determination unit includes:

14. The apparatus of claim 13, wherein the execution information acquisition subunit is configured to acquire at least one of the following execution information:

15. The apparatus of claim 14, wherein the coverage acquisition subunit comprises:

16. The apparatus according to claim 12, wherein the coverage test result determination unit includes:

17. The apparatus of claim 16, wherein the apparatus further comprises:

18. The apparatus of claim 17, wherein the second test unit comprises:

19. The apparatus of claim 18, wherein the supplemental test set generation subunit comprises:

and the notification second-level subunit is used for notifying the uploading person to generate a supplementary test set for the codes which are not covered by the test in the patch code segment according to the identification of the uploading person.

20. The apparatus of claim 12, wherein the apparatus further comprises:

21. The apparatus of claim 16, wherein the apparatus further comprises:

22. The apparatus of claim 21, wherein the apparatus further comprises:

23. The apparatus of claim 22, wherein the apparatus further comprises:

And the second operation quality determining unit is used for determining that the optimized patch code section passes the operation quality test if the third operation parameter reaches a third operation index threshold value.

24. An electronic device, comprising:

a memory and a processor;

If the tested coverage rate of the patch code section reaches the coverage rate threshold of the code section after the software code containing the patch code section is tested again, the method further comprises the following steps: calculating the overall coverage rate of the software code containing the patch code section; and if the overall coverage rate of the software code containing the patch code section reaches the overall coverage rate threshold, acquiring a first operation parameter generated when the software code containing the patch code section is tested and operated, wherein the first operation parameter is used for representing the operation quality of the software code containing the patch code section when the software code containing the patch code section is tested and operated.

25. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, performs the following operations: