CN117370145A

CN117370145A - Program testing method, device, electronic equipment and storage medium

Info

Publication number: CN117370145A
Application number: CN202210777379.1A
Authority: CN
Inventors: 杜兴文; 王哲; 谢晓群
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2022-07-01
Filing date: 2022-07-01
Publication date: 2024-01-09

Abstract

The disclosure provides a program testing method, a program testing device, electronic equipment and a storage medium, which can be applied to the technical field of software testing. The program test method comprises the following steps: in response to a test request for a target program, respectively inserting a probe field into a plurality of methods of a source code of the target program, wherein the probe field is configured with a statistic attribute, and the statistic attribute is used for recording the calling times of the method for inserting the probe field; based on the current test case, testing the target program to obtain statistical value attribute information of a plurality of probe fields; determining an uncovered method from the plurality of methods based on the statistical value attribute information of the plurality of probe fields; generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is greater than a preset threshold; and testing the target program based on the target test case.

Description

Program testing method, device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of software testing technology, and more particularly, to a program testing method, apparatus, electronic device, storage medium, and program product.

Background

Program testing is an active process of verifying and validating program products, and is an important component in the process of program development. The program product is tested, and various problems existing in the program product can be found in the development stage, so that the program product can be ensured to normally run after being on line.

As the iteration speed of the program product increases, so does the requirements for the efficiency of the program test. The program test method in the related art relies on the test experience of the tester, and generally uses a conventional case design method to perform the test. Therefore, the program testing method in the related art has the technical problems of low testing efficiency and high missing test rate.

Disclosure of Invention

In view of this, the present disclosure provides a program testing method, a program testing apparatus, an electronic device, a readable storage medium, and a computer program product.

One aspect of the present disclosure provides a program testing method, including: in response to a test request for a target program, respectively inserting a probe field into a plurality of methods of source codes of the target program, wherein the probe field is configured with a statistic attribute, and the statistic attribute is used for recording the calling times of the method for inserting the probe field; based on the current test case, testing the target program to obtain statistical value attribute information of a plurality of probe fields; determining an uncovered method from the plurality of methods based on statistical value attribute information of the plurality of probe fields; generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is larger than a preset threshold; and testing the target program based on the target test case.

According to an embodiment of the present disclosure, the above method further includes: determining a target historical version from a plurality of historical versions of the target program in a code base; determining a target increment method based on the source code of the target history version;

according to an embodiment of the present disclosure, inserting probe fields into the methods of the source code of the target program includes: the probe field is inserted into the target increment method of the target program.

According to an embodiment of the present disclosure, the method for determining a target increment based on the source code of the target history version includes: generating a first code tree based on source code of the target program, wherein the first code tree comprises a plurality of first nodes at a plurality of levels, each of the first nodes characterized as a method; generating a second code tree based on the target historical version source code, wherein the second code tree comprises a plurality of second nodes at a plurality of levels, each of the first nodes characterized as a method; comparing and analyzing the first code tree and the second code tree to determine an incremental method; and screening the increment method to obtain the target increment method.

According to an embodiment of the present disclosure, the delta method includes a difference method and a new method.

According to an embodiment of the present disclosure, the comparing and analyzing the first code tree and the second code tree to determine an incremental method includes: for each first node of the plurality of first nodes, determining identification information of the first node and position information of the first node in the first code tree; comparing the digest value of the method represented by the first node with the digest value of the method represented by the target second node to obtain a comparison result when it is determined that the target second node corresponding to the first node exists in the second code tree based on the identification information and the position information; determining that the method characterized by the first node is a difference method when the summary value of the method characterized by the first node and the summary value of the method characterized by the second node are inconsistent according to the comparison result; and determining that the method of the first node characterization is a new method when it is determined that the second node corresponding to the first node does not exist in the second code tree based on the identification information and the location information.

According to an embodiment of the present disclosure, the foregoing screening the foregoing increment method to obtain the foregoing target increment method includes: for each of the delta methods, determining configuration information of a file corresponding to the delta method; and determining the increment method as the target increment method under the condition that the configuration information of the file meets the preset condition.

According to an embodiment of the present disclosure, inserting probe fields into the methods of the source code of the target program includes: executing a preset instruction to start the agent program; and inserting the probe fields into a plurality of methods of source code of the target program using the agent program, respectively, before loading the target program using a class loader.

According to an embodiment of the present disclosure, the above method further includes: and generating a test report based on the uncovered method under the condition that the number of the uncovered methods is smaller than or equal to the preset threshold.

Another aspect of the present disclosure provides a program testing apparatus, comprising: the first processing module is used for respectively inserting probe fields into a plurality of methods of source codes of the target program in response to a test request for the target program, wherein the probe fields are configured with statistic value attributes, and the statistic value attributes are used for recording the calling times of the methods for inserting the probe fields; the first test module is used for testing the target program based on the current test case to obtain statistical value attribute information of a plurality of probe fields; a first determining module, configured to determine an uncovered method from the plurality of methods based on statistical value attribute information of a plurality of probe fields; the first generation module is used for generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is larger than a preset threshold value; and the second test module is used for testing the target program based on the target test case.

Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to implement the method as described above.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.

Another aspect of the present disclosure provides a computer program product comprising computer executable instructions which, when executed, are adapted to implement the method as described above.

According to the embodiment of the disclosure, by inserting the probe fields into the methods of the source code of the target program, the calling times of each method can be counted when the target program is tested by using the current test case, and then the uncovered method can be determined according to the counted statistical value attribute information, so that the test coverage rate of each method of the target program in the current test stage can be determined; then, under the condition that the test coverage rate does not meet the test requirement, a new target test case can be generated according to the uncovered method, and the target program can be continuously tested by using the target test case. By the technical means, the tested range can be definitely determined in the test process, so that repeated test on the tested part can be avoided in the subsequent test process, the technical problems of low test efficiency and high missing test rate of the program test method in the related technology are at least partially overcome, the coverage rate and the test efficiency of code test are effectively improved, and the test cost is saved.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:

FIG. 1 schematically illustrates an exemplary system architecture in which methods and apparatus for testing programs may be applied, according to embodiments of the present disclosure.

Fig. 2 schematically illustrates a flow chart of a program test method according to an embodiment of the disclosure.

Fig. 3 schematically illustrates a flow chart of a program test method according to another embodiment of the present disclosure.

Fig. 4 schematically illustrates a method flow diagram for a method of determining target increments in accordance with an embodiment of the disclosure.

Fig. 5 schematically illustrates a flow chart of a test report generation method according to an embodiment of the present disclosure.

Fig. 6 schematically illustrates an architecture diagram of a program test system according to an embodiment of the present disclosure.

Fig. 7 schematically illustrates a block diagram of a program testing apparatus according to an embodiment of the present disclosure.

Fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a program test method according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a formulation similar to at least one of "A, B or C, etc." is used, in general such a formulation should be interpreted in accordance with the ordinary understanding of one skilled in the art (e.g. "a system with at least one of A, B or C" would include but not be limited to systems with a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the related art test scenario, the test types of the program test may include a functional test, a regression test, an interface test, and the like, and the above-mentioned types of tests are generally full-flow tests, that is, a full-volume verification of the program code by a tester is required. Along with the development of agile development, the development iteration speed of the program product is faster and faster, and the business logic of the program product is more and more complex, so that the whole process test needs to return to all test cases after each iteration of the program product, and the waste of test resources is caused.

On the other hand, since the program development stage and the program testing stage are generally performed by different technicians, the testers and developers often have differences in understanding the program product, so that the testing accuracy and coverage rate of the testing method depending on the testing experience of the testers are difficult to ensure, and thus the testing efficiency of the program test is low.

In view of this, the embodiments of the present disclosure provide a program testing method, which can accurately evaluate the uncovered range of the test, save the testing resources, and improve the testing efficiency. Specifically, embodiments of the present disclosure provide a program testing method, a program testing apparatus, an electronic device, a readable storage medium, and a computer program product. The method comprises the following steps: in response to a test request for a target program, respectively inserting a probe field into a plurality of methods of a source code of the target program, wherein the probe field is configured with a statistic attribute, and the statistic attribute is used for recording the calling times of the method for inserting the probe field; based on the current test case, testing the target program to obtain statistical value attribute information of a plurality of probe fields; determining an uncovered method from the plurality of methods based on the statistical value attribute information of the plurality of probe fields; generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is greater than a preset threshold; and testing the target program based on the target test case.

FIG. 1 schematically illustrates an exemplary system architecture in which methods and apparatus for testing programs may be applied, according to embodiments of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105.

The terminal devices 101, 102, 103 may be various electronic devices with display screens including, but not limited to, tablet computers, laptop portable computers, desktop computers, and the like.

Various client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients and/or social platform software, etc., may be loaded on the terminal devices 101, 102, 103.

The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like. A user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103.

The server 105 may be a server providing various services, for example, the server 105 may be a background server providing support for program testing operations performed by the user using the terminal devices 101, 102, 103, and the background server may use the received test cases to test a program to be tested and feed back test results to the terminal devices.

It should be noted that the program testing method provided by the embodiments of the present disclosure may be generally executed by the server 105. Accordingly, the program testing apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 105. The program testing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the program testing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Alternatively, the program testing method provided by the embodiment of the present disclosure may be executed by the terminal apparatus 101, 102, or 103, or may be executed by another terminal apparatus other than the terminal apparatus 101, 102, or 103. Accordingly, the program testing apparatus provided by the embodiments of the present disclosure may also be provided in the terminal device 101, 102, or 103, or in another terminal device different from the terminal device 101, 102, or 103.

For example, the test cases and the target programs may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, but not limited to, the terminal device 101), or stored on an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the program testing method provided by the embodiment of the present disclosure in response to the test request, or send the test case and the target program to other terminal devices, servers, or server clusters, and after receiving the test request, execute the program testing method provided by the embodiment of the present disclosure by other terminal devices, servers, or server clusters that receive the test case and the target program.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

As shown in fig. 2, the method includes operations S201 to S205.

It should be noted that, unless there is an execution sequence between different operations or an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may be different, and multiple operations may also be executed simultaneously in the embodiment of the disclosure.

In operation S201, probe fields are inserted into a plurality of methods of source code of a target program, respectively, in response to a test request for the target program. The probe field is configured with a statistics attribute for recording the number of invocations of the method of inserting the probe field.

In operation S202, based on the current test case, the target program is tested to obtain statistical value attribute information of a plurality of probe fields.

In operation S203, an uncovered method is determined from among the plurality of methods based on the statistical value attribute information of the plurality of probe fields.

In operation S204, in case the number of uncovered methods is greater than a preset threshold, a target test case is generated based on the uncovered methods.

In operation S205, a test is performed on the target program based on the target test case.

According to an embodiment of the present disclosure, the target program may be a program to be tested in a development flow. The target program may be various client applications such as a shopping class application, a communication class application, a search class application, and the like. The source code of the target program may refer to the source code of the version to be tested in the current development flow. The program language used to compile the source code may be JAVA, python, C ++, for example, and is not limited thereto.

According to embodiments of the present disclosure, the probe field may be a string of codes that can record the number of calls when called, without affecting the normal operation of the method. The programming language used to compile the probe fields may be the same as the programming language used when compiling the source code.

According to an embodiment of the present disclosure, the statistic attribute may be a variable defined in the probe field, and the value of the variable is statistic attribute information. The statistics attribute may record the number of times the probe field was invoked, and the value of a specific invocation may record the value of the statistics attribute, i.e., statistics attribute information.

According to the embodiment of the disclosure, the current test cases may include test cases that are compiled by a tester in advance based on an application scenario of the target program, or may include test cases that are randomly generated based on input/output characteristics of the target program, which is not limited herein.

According to the embodiment of the disclosure, the test of the target program based on the current test case can be understood as that the current test case is input from the external interface of the target program to realize the test of the target program.

According to embodiments of the present disclosure, the uncovered method may be a method that has not been invoked during the test. That is, it is possible to determine that among the statistic attribute information of the plurality of probe fields, the method corresponding to the probe field having the statistic attribute information of 0 is an uncovered method.

According to the embodiment of the disclosure, the preset threshold value can be set according to the requirement for program testing in a specific application scenario. For example, the preset threshold may be set to a fixed value, such as 50, 100, etc. For another example, the preset threshold may be set to a value proportional to the total number of methods of the target program, such as 5%, 10%, or the like of the total number of methods of the target program.

According to the embodiment of the disclosure, the number of uncovered methods being greater than the preset threshold may be understood as that the current test case fails to meet the requirement for the program test in the current scenario, or may be understood as that the code coverage rate of the current test process fails to meet the test requirement. Code coverage can be understood as the ratio of the tested method to the total number of methods.

According to embodiments of the present disclosure, generating a target test case based on an uncovered method may be understood as: based on the function of the uncovered method in the target program, determining a functional module associated with the uncovered method, and generating a target test case according to the input-output characteristics of the functional module.

According to the embodiment of the disclosure, after the target program is tested by using the target test case, the uncovered method under the current condition can be determined again, and then a new test case is generated according to the uncovered method under the current condition to test, so that accurate test is realized.

The method shown in fig. 2 is further described below with reference to fig. 3-6, in conjunction with the exemplary embodiment.

According to an embodiment of the present disclosure, operation S201 may include the following operations:

executing a preset instruction to start the agent program; and, before loading the target program using the class loader, inserting probe fields in a plurality of methods of source code of the target program, respectively, using the agent program.

According to embodiments of the present disclosure, the insertion of the probe fields may be implemented using a code coverage tool, such as Jacoco, emma, cobertura. Taking Jacoco as an example for inserting source codes of target programs, a JVM can be used for specifying jar files through a java agent parameter to start an agent program, and then the agent program can insert probes into class files before a class is loaded by a class loader.

In other embodiments of the present disclosure, the test may be performed on the incremental code in the target program without using a full test method, so as to implement an accurate test on the target program and save test resources.

As shown in fig. 3, the method includes operations S301 to S308.

In operation S301, a target history version is determined from a plurality of history versions of a target program in a code library in response to a test request for the target program.

In operation S302, a target delta method is determined based on source code of a target history version.

In operation S303, a probe field is inserted into a target delta method of a target program.

In operation S304, based on the first test case, the target program is tested to obtain statistical value attribute information of the probe field.

In operation S305, an uncovered method is determined from the target delta methods based on the statistical value attribute information of the probe field.

In operation S306, it is determined whether the number of uncovered methods is greater than a preset threshold. In case it is determined that the number of uncovered methods is greater than the preset threshold, operation S307 is performed; in case it is determined that the number of uncovered methods is less than or equal to the preset threshold, operation S308 is performed.

In operation S307, a second test case is generated based on the uncovered method, and the second test case is added to the first test case. After the operation S307 is completed, the execution of operation S304 may be returned.

In operation S308, a test report is generated based on the uncovered method.

According to an embodiment of the present disclosure, the code library may be a database for storing source code of historical versions of the target program; or a folder for storing source code of historical versions of the target program, each of which may be stored in a subfolder under the folder directory.

According to embodiments of the present disclosure, a historical version may refer to an online version of a target program. The historical versions of the target program stored in the code base may include a full number of online versions, or may include only a limited number of recently online versions, which is not limited herein.

According to embodiments of the present disclosure, among a plurality of history versions of a code library, a target history version may refer to a history version having the shortest on-line time, i.e., a last on-line version of a target program; reference may also be made to a baseline version, i.e., the last stable version of the target program, without limitation.

According to an embodiment of the present disclosure, the target delta method may include a method of adding a new version of the target program compared to the target history version, and may also include a method of having a difference in the current version compared to the target history version. The determination of the target increment method may be implemented by a code comparison tool such as WinMerge, diffuse, codeCompare, or may be implemented by a method of compiling the source code into a call chain, a semantic tree, or the like, which is not limited herein.

According to an embodiment of the disclosure, the first test case may be generated by determining a function module associated with the target increment method based on a function of the target increment method in the target program, and then according to an input-output characteristic of the function module. Alternatively, the first test case may be a test case associated with the functional module in the original test cases.

According to the embodiment of the disclosure, the number of uncovered methods being less than or equal to the preset threshold value may be understood as that the current test case can meet the requirement for program testing in the current scenario, or may be understood as that the code coverage rate of the current test process can meet the test requirement.

According to the embodiment of the disclosure, the test report may further include information such as code coverage rate, test result, and the like, in addition to the uncovered method. After the test report is generated, the test report can be returned to a developer, and the developer can position the code position of the bug (bug) and the affected functional range according to the test report, so that the association between test data and the developed code is realized, the optimization of the target program is realized, and the test efficiency is improved.

According to the embodiment of the disclosure, the test cases for testing are determined according to the incremental method by locating the incremental method, so that the targeted test of the target program can be realized, the regression test range can be more accurate, the time required by the regression test is reduced, and the test efficiency is improved.

According to an embodiment of the present disclosure, the determination of the target delta method in operation S302 may be implemented by means of a code tree. The method of operation S302 is further described below with reference to fig. 4 in conjunction with the exemplary embodiment.

As shown in fig. 4, the method includes operations S401 to S404.

In operation S401, a first code tree is generated based on source code of a current version of a target program.

In operation S402, a second code tree is generated based on the target history version source code.

In operation S403, a comparison analysis is performed on the first code tree and the second code tree to determine an incremental method.

In operation S404, the increment method is filtered to obtain a target increment method.

According to an embodiment of the present disclosure, the first code tree or the second code tree may be a method call chain represented as a tree structure, or may be an abstract syntax tree. The code tree includes a plurality of nodes at a plurality of levels, each node characterized as a method. The generation of the first code tree or the second code tree may be implemented using any call chain generation tool or abstract syntax tree generation tool. For example, the call chain of a JAVA program may be generated using IDEA, jaeger, etc.; as another example, an abstract syntax tree of a JAVA program may be generated using eclipse-astparser, javaparser or the like.

According to embodiments of the present disclosure, the first code tree and the second code tree may be compared using a Git or other tool to determine an incremental approach.

According to embodiments of the present disclosure, the delta methods may include a difference method and a new method.

According to an embodiment of the present disclosure, a comparison analysis of a first code tree and a second code tree to determine an incremental method may include the operations of:

for each first node of the plurality of first nodes, determining identification information of the first node and location information of the first node in the first code tree; under the condition that a second node corresponding to the first node exists in the second code tree based on the identification information and the position information, comparing the abstract value of the method represented by the first node with the abstract value of the method represented by the second node to obtain a comparison result; determining that the method characterized by the first node is a difference method under the condition that the digest value of the method characterized by the first node and the digest value of the method characterized by the second node are inconsistent; and determining that the method of the first node characterization is a new method in the case that the second node corresponding to the first node does not exist in the second code tree based on the identification information and the position information.

According to an embodiment of the present disclosure, the identification information of the first node may be a name of a method corresponding to the first node. The location information of the first node may include information of a hierarchy in which the first node is located in the first code tree, and information of upper and lower nodes associated with the first node, that is, call relationship information.

According to embodiments of the present disclosure, the target second node may refer to a second node having the same identification information and location information as the first node. When searching for the target second node from the second code tree based on the identification information and the position information of the first node, whether the second node with the same level information and the same calling relation information as the first node exists in the second code tree can be searched according to the position information of the first node. In the case where it is determined that the second node exists, it is possible to consider that the target second node exists in the second code tree by checking whether the name of the method represented by the first node and the name of the method represented by the second node coincide, and in the case where it is determined that the method names coincide.

According to embodiments of the present disclosure, the digest value may be obtained by processing the source code of the method using a digest algorithm such as MD5, SHA, or the like. The digest value may be represented as a string. When the digest value of the method represented by the first node and the digest value of the method represented by the second node are compared, each character of the two character strings can be compared in turn, and when a certain character in the two character strings is inconsistent, the digest value of the method represented by the first node and the digest value of the method represented by the second node can be considered to be inconsistent.

According to the embodiment of the disclosure, when the digest value of the method represented by the first node and the digest value of the method represented by the second node are consistent, the method represented by the first node may be considered to be unchanged in the current development stage, and the method represented by the first node may not be tested.

According to the embodiment of the disclosure, after the difference method and the new method are determined, the difference method and the new method may be marked, that is, a method with marks in a plurality of methods of the target program is an incremental method.

According to embodiments of the present disclosure, after determining the delta method, the delta method may also be screened to determine a target delta method that requires probe field instrumentation. Specifically, for each of the delta methods, configuration information of a file corresponding to the delta method may be determined; and determining the increment method as a target increment method under the condition that the configuration information of the file meets the preset condition.

According to embodiments of the present disclosure, the configuration information of the file may include information of a type of the file, a path of the file, and the like.

According to the embodiment of the disclosure, the preset condition may be a constraint condition of a file to which the method belongs, and the preset condition may be set according to a specific application scenario. For example, the preset condition may be set to include a file whose file type is java type, and a path of the file is a main file directory.

According to the embodiment of the disclosure, the test cases affected by the code change can be accurately evaluated by determining the increment method, so that accurate and targeted test can be conveniently performed, the time required by regression test is reduced, and the regression function range is more accurate.

According to an embodiment of the present disclosure, the generation of the test report in operation S308 may be implemented using a code coverage tool such as Jacoco, emma, cobertura. The method of operation S308 will be further described with reference to fig. 5, taking Jacoco as an example.

As shown in fig. 5, the method includes operations S501 to S506.

In operation S501, a test service is deployed using a java agent.

In operation S502, a test case is executed, and an uncovered method and a test result are obtained.

In operation S503, a test report type is determined. In case that it is determined that the test report type is the delta type, operation S504 is performed; in the case where it is determined that the test report type is the full-quantity type, operation S505 is performed.

In operation S504, an incremental method among the uncovered methods is determined as a target method.

In operation S505, the uncovered method is determined as the target method.

After completing operation S504 or S505, operation S506 may be performed.

In operation S506, a test report is generated based on the target method and the test result.

According to embodiments of the present disclosure, the test report may be generated using the modified jacococli. The modified jacococli.jar command may include a parameter for specifying the type of test report. Specifically, the modification method of the jacococli.jar command may include: the modify analysis method passes the delta code to the convergebuilder object and modifies the visittmethod of the ClassProbesAdapter class.

In some embodiments, the test report types may also include a full+incremental type, and the test report generated according to the type may include a test report generated based on the methods of operations S504 and S506, and a test report generated based on the methods of operations S505 and S506.

As shown in fig. 6, in the development phase of a program, source code 602 developed by a developer 601 may be stored in a code repository 603.

Prior to testing, the server cluster 604 may implement a compilation build of source code 602 in the code repository 603 to compile the source code 602 into an application 605 for testing. In the compiling and constructing process, the probe fields can be inserted into the methods of the source code in an inserting mode.

During the testing phase of the program, a tester 606 may install the application 605 in the test equipment 607 to facilitate testing of the application 605 using test cases. The test equipment 607 may generate new test cases according to the code coverage rate in the test process and continue the test. After testing is completed, the test equipment 607 may generate a test report 608 based on the test coverage and the test results. The test report may be returned to developer 601. The developer 601 can realize operations such as locating the code position of the bug, mining the missing test case, locating the functional range of influence and the like according to the test report, thereby forming new source codes, test cases and functional requirements, and developing the source codes 602 again.

According to the embodiment of the disclosure, by using the system, a developer can be assisted to accurately evaluate the test range, the solving efficiency of the code loopholes is improved, the tester can be assisted to conduct targeted tests, the test risk is reduced, and the repeated labor is reduced, so that the efficiency of program development is improved as a whole.

As shown in fig. 7, the program testing apparatus 700 includes a processing module 710, a first testing module 720, a first determining module 730, a first generating module 740, and a second testing module 750.

A processing module 710, configured to insert probe fields into a plurality of methods of source code of a target program in response to a test request for the target program, respectively. The probe field is configured with a statistics attribute for recording the number of invocations of the method of inserting the probe field.

The first test module 720 is configured to test the target program based on the current test case, and obtain statistical value attribute information of the plurality of probe fields.

A first determining module 730, configured to determine an uncovered method from a plurality of methods based on statistical value attribute information of the plurality of probe fields.

The first generating module 740 is configured to generate, based on the uncovered methods, a target test case if the number of uncovered methods is greater than a preset threshold.

And a second test module 750, configured to test the target program based on the target test case.

According to an embodiment of the present disclosure, the program testing device 700 further includes a second determination module and a third determination module.

And the second determining module is used for determining a target historical version from a plurality of historical versions of the target program in the code base.

And a third determining module, configured to determine a target delta method based on the source code of the target history version.

According to an embodiment of the present disclosure, the processing module 710 includes a first processing unit.

And the first processing unit is used for inserting the probe field into the target increment method of the target program.

According to an embodiment of the present disclosure, the third determination module includes a first determination unit, a second determination unit, a third determination unit, and a fourth determination unit.

A first determining unit for generating a first code tree based on the source code of the target program, wherein the first code tree comprises a plurality of first nodes at a plurality of levels, each first node being characterized as a method.

And a second determining unit, configured to generate a second code tree based on the target historical version source code, where the second code tree includes a plurality of second nodes located at a plurality of levels, and each of the first nodes is characterized as a method.

And the third determining unit is used for comparing and analyzing the first code tree and the second code tree to determine an increment method.

And the fourth determining unit is used for screening the increment method to obtain the target increment method.

According to embodiments of the present disclosure, the delta methods include a difference method and a new method.

According to an embodiment of the present disclosure, the third determination unit includes a first determination subunit, a second determination subunit, a third determination subunit, and a fourth determination subunit.

A first determining subunit configured to determine, for each of the plurality of first nodes, identification information of the first node and location information of the first node in the first code tree.

And the second determining subunit is used for comparing the digest value of the method represented by the first node with the digest value of the method represented by the target second node to obtain a comparison result when determining that the target second node corresponding to the first node exists in the second code tree based on the identification information and the position information.

And the third determining subunit is used for determining that the method characterized by the first node is a difference method when the digest value of the method characterized by the first node and the digest value of the method characterized by the second node are inconsistent.

And a fourth determining subunit, configured to determine, when it is determined, based on the identification information and the location information, that the second node corresponding to the first node does not exist in the second code tree, that the method for determining the first node representation is a new method.

According to an embodiment of the present disclosure, the fourth determination unit includes a fifth determination subunit and a sixth determination subunit.

And a fifth determination subunit configured to determine, for each of the incremental methods, configuration information of a file corresponding to the incremental method.

And a sixth determining subunit, configured to determine the incremental method as the target incremental method if the configuration information of the file satisfies the preset condition.

According to an embodiment of the present disclosure, the processing module 710 includes a second processing unit and a third processing unit.

And the second processing unit is used for executing a preset instruction to start the agent program.

And a third processing unit for inserting probe fields in a plurality of methods of source code of the target program, respectively, using the agent program before loading the target program using the class loader.

According to an embodiment of the present disclosure, the program testing apparatus 700 further includes a second generation module.

And the second generation module is used for generating a test report based on the uncovered methods under the condition that the number of the uncovered methods is smaller than or equal to a preset threshold.

Any number of modules, sub-modules, units, sub-units, or at least some of the functionality of any number of the sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in any other reasonable manner of hardware or firmware that integrates or encapsulates the circuit, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

For example, any of the processing module 710, the first test module 720, the first determination module 730, the first generation module 740, and the second test module 720 may be combined in one module/unit/sub-unit or any of the modules/units/sub-units may be split into a plurality of modules/units/sub-units. Alternatively, at least some of the functionality of one or more of these modules/units/sub-units may be combined with at least some of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to embodiments of the present disclosure, one of the processing module 710, the first testing module 720, the first determination module 730, the first generation module 740, and the second testing module 720 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of any of the three. Alternatively, one of the processing module 710, the first test module 720, the first determination module 730, the first generation module 740, and the second test module 720 may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

It should be noted that, in the embodiment of the present disclosure, the program testing device portion corresponds to the program testing method portion in the embodiment of the present disclosure, and the description of the program testing device portion specifically refers to the program testing method portion and is not repeated herein.

Fig. 8 schematically illustrates a block diagram of an electronic device adapted to implement a program test method according to an embodiment of the disclosure. The electronic device shown in fig. 8 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 8, a computer electronic device 800 according to an embodiment of the present disclosure includes a processor 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. The processor 801 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the disclosure.

In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 802 and/or the RAM 803. Note that the program may be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 800 may also include an input/output (I/O) interface 805, the input/output (I/O) interface 805 also being connected to the bus 804. The electronic device 800 may also include one or more of the following components connected to the I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.

According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 802 and/or RAM 803 and/or one or more memories other than ROM 802 and RAM 803 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program comprising program code for performing the methods provided by the embodiments of the present disclosure, when the computer program product is run on an electronic device, for causing the electronic device to implement the program testing methods provided by the embodiments of the present disclosure.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, and/or from a removable medium 811 via a communication portion 809. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A program testing method, comprising:

in response to a test request for a target program, respectively inserting a probe field into a plurality of methods of source codes of the target program, wherein the probe field is configured with a statistic attribute, and the statistic attribute is used for recording the calling times of the method for inserting the probe field;

based on the current test case, testing the target program to obtain statistical value attribute information of a plurality of probe fields;

determining an uncovered method from the plurality of methods based on statistical value attribute information of a plurality of the probe fields;

generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is larger than a preset threshold; and

And testing the target program based on the target test case.

2. The method of claim 1, further comprising:

determining a target historical version from a plurality of historical versions of the target program in a code base;

determining a target increment method based on the source code of the target history version;

the method for inserting the probe fields into the source code of the target program respectively comprises the following steps:

inserting the probe field into the target delta method of the target program.

3. The method of claim 2, wherein the determining a target delta method based on the source code of the target historical version comprises:

generating a first code tree based on source code of the target program, wherein the first code tree comprises a plurality of first nodes at a plurality of levels, each first node characterized by a method;

generating a second code tree based on the target historical version source code, wherein the second code tree comprises a plurality of second nodes at a plurality of levels, each first node characterized as a method;

comparing and analyzing the first code tree and the second code tree to determine an incremental method; and

And screening the increment method to obtain the target increment method.

4. A method according to claim 3, wherein the incremental methods include a difference method and a new method;

wherein said comparing said first code tree and said second code tree to determine an incremental method comprises:

for each first node of the plurality of first nodes, determining identification information of the first node and location information of the first node in the first code tree;

comparing the digest value of the method represented by the first node with the digest value of the method represented by the target second node under the condition that the target second node corresponding to the first node exists in the second code tree based on the identification information and the position information, so as to obtain a comparison result;

determining that the method characterized by the first node is a difference method under the condition that the digest value of the method characterized by the first node and the digest value of the method characterized by the second node are inconsistent; and

and determining that the method for characterizing the first node is a new method under the condition that the second node corresponding to the first node does not exist in the second code tree based on the identification information and the position information.

5. A method according to claim 3, wherein said screening of said delta method results in said target delta method comprising:

for each of the incremental methods, determining configuration information of a file corresponding to the incremental method; and

and under the condition that the configuration information of the file meets the preset condition, determining the increment method as the target increment method.

6. The method of claim 1, wherein the inserting probe fields into the methods of the source code of the target program includes:

executing a preset instruction to start the agent program; and

the probe fields are inserted in a plurality of methods of source code of the target program, respectively, using the agent program before loading the target program using a class loader.

7. The method of claim 1, further comprising:

and generating a test report based on the uncovered methods under the condition that the number of the uncovered methods is smaller than or equal to the preset threshold.

8. A program testing method, comprising:

the first processing module is used for respectively inserting a probe field into a plurality of methods of a source code of a target program in response to a test request for the target program, wherein the probe field is configured with a statistic attribute, and the statistic attribute is used for recording the calling times of the method for inserting the probe field;

The first test module is used for testing the target program based on the current test case to obtain statistical value attribute information of a plurality of probe fields;

a first determining module, configured to determine an uncovered method from the plurality of methods based on statistical value attribute information of a plurality of the probe fields;

the first generation module is used for generating a target test case based on the uncovered methods under the condition that the number of the uncovered methods is larger than a preset threshold value; and

and the second test module is used for testing the target program based on the target test case.

9. An electronic device, comprising:

one or more processors;

a memory for storing one or more instructions,

wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 7.

10. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 7.

11. A computer program product comprising computer executable instructions for implementing the method of any one of claims 1 to 7 when executed.