CN110580214B - Evaluation method and device for application program test, storage medium and terminal equipment - Google Patents
Evaluation method and device for application program test, storage medium and terminal equipment Download PDFInfo
- Publication number
- CN110580214B CN110580214B CN201810581509.8A CN201810581509A CN110580214B CN 110580214 B CN110580214 B CN 110580214B CN 201810581509 A CN201810581509 A CN 201810581509A CN 110580214 B CN110580214 B CN 110580214B
- Authority
- CN
- China
- Prior art keywords
- coverage
- test
- application program
- gray
- point set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides an evaluation method, an evaluation device, a storage medium and terminal equipment for application program testing, wherein the method comprises the following steps: according to a test behavior log of an application program in an online stage, counting an online coverage point set and an online test coverage rate of the application program; according to the user behavior log of the application program in the gray level stage, counting a gray level coverage point set and gray level test coverage rate of the application program; according to the off-line coverage point set and the gray coverage point set, counting the overall test coverage rate; and evaluating the test result of the application program according to the off-line test coverage rate, the gray scale test coverage rate and the overall test coverage rate. By adopting the method and the device, the test result of the whole test process of the application program before formal release can be evaluated.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an evaluation method and apparatus for application testing, a storage medium, and a terminal device.
Background
Coverage is used for measuring the coverage degree of a currently executed test case on a test object, and is a measure for measuring the integrity and the effectiveness of the test case.
At present, the implementation mode of the test coverage rate of the statistical application program comprises the following schemes:
1. the test coverage was counted using the AOP (Aspect Oriented Programming, tangent plane oriented programming) method. AOP is a technique that dynamic agents implement unified maintenance of program functions during the running of an application by means of precompiled. The AOP inserts the overlay mark code into the original code of the application program, and the application program can record the overlay information which is executed in the executing process without affecting the executing behavior of the original code. Further, according to the recorded coverage information to be executed, the test coverage rate of the application program is counted.
2. The code of coverage statistics is added in the code of the application program in a code instrumentation mode, and the test coverage can be directly counted in the running process of the application program.
In the statistics process of the test coverage of the two schemes, statistics can be performed according to coverage conditions such as function coverage, statement coverage, judgment coverage, condition coverage, path coverage and the like of the application program test. However, such covered data is mostly used for the execution effect of single test sets, such as full function regression, statistics of test coverage of new function tests. For an application to be published, it is difficult to give an overall assessment of the entire project in which the application was developed.
Disclosure of Invention
The embodiment of the invention provides an evaluation method, an evaluation device, a storage medium and terminal equipment for application program testing, which are used for solving or relieving one or more of the technical problems in the prior art.
In a first aspect, an embodiment of the present invention provides an evaluation method for application testing, including:
according to a test behavior log of an application program in an online stage, counting an online coverage point set and an online test coverage rate of the application program;
according to the user behavior log of the application program in the gray level stage, counting a gray level coverage point set and gray level test coverage rate of the application program;
according to the off-line coverage point set and the gray coverage point set, counting the overall test coverage rate;
and evaluating the test result of the application program according to the off-line test coverage rate, the gray scale test coverage rate and the overall test coverage rate.
With reference to the first aspect, in a first implementation manner of the first aspect, the counting, according to a test behavior log of an application program in an online offline stage, an offline coverage point set and an offline test coverage rate of the application program includes:
according to the test behavior logs of each off-line test of the on-line stage of the application program, counting a corresponding test coverage point set of each off-line test; the elements of the test coverage point set represent the operation behavior of the application program tested in a single offline test;
The elements contained in the test coverage point sets of each off-line test are combined to obtain an off-line coverage point set of the application program;
and obtaining the offline test coverage rate of the application program according to the offline coverage point set and the buried point complete set of the application program, wherein the offline test coverage rate is the ratio of the number of elements of the offline coverage point set to the number of elements of the buried point complete set, and the elements of the buried point complete set represent all the operable behaviors of the application program.
With reference to the first aspect, in a second implementation manner of the first aspect, the counting, according to the user behavior log of the application program in the gray level stage, a gray level coverage point set and a gray level test coverage rate of the application program includes:
according to the user behavior log of each user using the application program in the gray level stage, counting a corresponding use coverage point set of each user; the elements of the set of usage coverage points represent the operational behavior of a single user running the application;
the elements contained in the using coverage point set of each user are combined to obtain a gray coverage point set of the application program;
and obtaining the gray test coverage rate of the application program according to the gray coverage point set and the buried point total set of the application program, wherein the gray test coverage rate is the ratio of the number of elements of the gray coverage point set to the number of elements of the buried point total set, and the elements of the buried point total set represent all the operable behaviors of the application program.
With reference to the first aspect, in a third implementation manner of the first aspect, the counting the overall test coverage according to the offline coverage point set and the gray coverage point set includes:
the elements in the offline coverage point set and the gray coverage point set are combined to obtain an overall coverage point set of the application program;
and obtaining the overall test coverage rate of the application program according to the overall coverage point set and the embedded point total set of the application program, wherein the overall test coverage rate is the ratio between the element number of the overall coverage point set and the element number of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program.
With reference to the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the method further includes:
and screening out operation behaviors which are not covered by the test from operation behaviors contained in the application program according to the whole covered point set of the application program.
With reference to the first aspect or any implementation manner thereof, in a fifth implementation manner of the first aspect, the offline stage includes a stage of testing the application before the application is not released to a user for use; and the gray scale stage comprises a stage that the application program is opened for a part of users to use.
In a second aspect, an embodiment of the present invention provides an evaluation apparatus for application testing, including:
the off-line coverage statistics module is used for counting an off-line coverage point set and an off-line test coverage rate of the application program according to a test behavior log of the application program at an off-line stage;
the gray coverage statistics module is used for counting gray coverage point sets and gray test coverage rate of the application program according to the user behavior log of the application program in the gray stage;
the integral coverage statistics module is used for counting integral test coverage according to the off-line coverage point set and the gray coverage point set; and
and the test result evaluation module is used for evaluating the test result of the application program according to the off-line test coverage rate, the gray level test coverage rate and the overall test coverage rate.
With reference to the second aspect, in a first implementation manner of the second aspect, the offline coverage statistics module includes:
the test coverage point statistics unit is used for counting a test coverage point set corresponding to each offline test according to the test behavior log of each offline test of the online offline stage of the application program; the elements of the test coverage point set represent the operation behavior of the application program tested in a single offline test;
The off-line coverage statistics unit is used for merging elements contained in the test coverage point sets of each off-line test to obtain an off-line coverage point set of the application program;
the off-line coverage rate calculation unit is used for obtaining the off-line test coverage rate of the application program according to the off-line coverage point set and the buried point total set of the application program, wherein the off-line test coverage rate is the ratio of the number of elements of the off-line coverage point set to the number of elements of the buried point total set, and the elements of the buried point total set represent all the operable behaviors of the application program.
With reference to the second aspect, in a second implementation manner of the second aspect, the gray coverage statistics module includes:
the user coverage point statistics unit is used for counting a corresponding usage coverage point set of each user according to the user behavior log of each user using the application program in the gray level stage; the elements of the set of usage coverage points represent the operational behavior of a single user running the application;
the gray coverage statistics unit is used for merging elements contained in the coverage point set of each user to obtain a gray coverage point set of the application program; and
the gray coverage rate calculation unit is used for obtaining gray test coverage rate of the application program according to the gray coverage point set and the embedded point total set of the application program, wherein the gray test coverage rate is a ratio of the number of elements of the gray coverage point set to the number of elements of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program.
With reference to the second aspect, in a third implementation manner of the second aspect, the overall coverage statistics module includes:
the integral coverage point statistics unit is used for merging the elements in the offline coverage point set and the gray coverage point set to obtain an integral coverage point set of the application program;
the whole coverage rate calculation unit is used for obtaining the whole test coverage rate of the application program according to the whole coverage point set and the embedded point whole set of the application program, wherein the whole test coverage rate is the ratio of the element number of the whole coverage point set to the element number of the embedded point whole set, and the elements of the embedded point whole set represent all the operable behaviors of the application program.
The functions of the device can be realized by hardware, and also can be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above.
In one possible design, the evaluation structure of the application test includes a processor and a memory, where the memory is configured to execute the evaluation program of the application test in the first aspect, and the processor is configured to execute the program stored in the memory. The evaluation means of the application test may further comprise a communication interface for communication of the evaluation means of the application test with other devices or with a communication network.
In a third aspect, an embodiment of the present invention further provides a computer readable storage medium storing computer software instructions for use by an evaluation device for application testing, including a program for executing the evaluation method for application testing of the first aspect.
One of the above technical solutions has the following advantages or beneficial effects:
according to the embodiment of the invention, corresponding offline test coverage rate and gray test coverage rate are respectively counted according to the test behavior log of the online offline stage of the application program and the user behavior log of the gray stage. According to the offline and gray levels of the application program and the overall test coverage rate, the test result of the whole test process of the application program before formal release can be evaluated. Thus, coverage closer to the business description than code coverage can be obtained. Further, since the behavior log can count the state of the core business of the application program, the behavior log is used for calculating the test coverage rate, and the cost for counting the test coverage rate is low.
The other technical scheme has the following advantages or beneficial effects: in the test process, uncovered points can be screened out and examined according to the coverage rate, and the application program can be subjected to supplementary test in an off-line test supplementary or corresponding gray level user adding mode.
The foregoing summary is for the purpose of the specification only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will become apparent by reference to the drawings and the following detailed description.
Drawings
In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.
FIG. 1 is a flow chart of one embodiment of an evaluation method for application testing provided by the present invention;
FIG. 2 is a flow diagram of one embodiment of a statistical process for offline test coverage provided by the present invention;
FIG. 3 is a flow chart of one embodiment of a statistical process for gray test coverage provided by the present invention;
FIG. 4 is a flow diagram of one embodiment of a statistical process for overall test coverage provided by the present invention;
FIG. 5 is a schematic diagram of an application example of an overlay case of application testing provided by the present invention;
FIG. 6 is a schematic diagram illustrating the architecture of an embodiment of an evaluation apparatus for application testing provided by the present invention;
FIG. 7 is a schematic diagram of another embodiment of an evaluation apparatus for application testing according to the present invention;
fig. 8 is a schematic structural diagram of an embodiment of a terminal device provided by the present invention.
Detailed Description
Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.
Referring to fig. 1, an embodiment of the present invention provides an evaluation method for application testing, which can be applied to a server. The present embodiment includes steps S100 to S400, which are specifically as follows:
s100, according to the test behavior log of the application program in the online stage, counting an online coverage point set and an online test coverage rate of the application program.
S200, counting gray coverage point sets and gray test coverage rate of the application program according to user behavior logs of the application program in a gray stage.
S300, counting the overall test coverage rate according to the offline coverage point set and the gray coverage point set.
S400, evaluating the test result of the application program according to the off-line test coverage, the gray scale test coverage and the overall test coverage.
In the embodiment of the present invention, all pages in the application program and all operational behaviors in the pages may be embedded in advance, for example: the test code is embedded in an AOP mode, so that the application program has coverage evaluable performance.
In the embodiment of the invention, the application program comprises three stages in the whole project development process: an off-line phase, a gray-scale phase, and an on-line phase. The test procedure of the application may include an off-line phase and a gray-scale phase. The offline phase may be referred to as a development phase of the application program, and is a phase of testing the application program before the application program is not released to a user for use. The offline stage can make corresponding functional tests on the application program based on the work of offline quality assessment, including functional tests with different granularities of units, modules, integration, etc. in the application program. Accordingly, the offline functional test may be evaluated according to the offline test coverage. The gray scale phase may be referred to as a public test phase of the application program, which is a phase in which the application program is opened for use by a part of users. For example, an application is opened to a small percentage of users for trial and experience, where problems with the application are discovered. For the quality evaluation work of the gray level stage, the evaluation mode of the part is mainly counted according to the automatically recovered user behavior log. The gray test coverage rate is calculated based on the user behavior log, and the coverage condition of the function of the application program used by the user can be estimated according to the gray test coverage rate. The online phase generally refers to a phase after the formal release of software.
In the embodiment of the invention, corresponding offline test coverage rate and gray test coverage rate are respectively counted according to the test behavior log of the online offline stage of the application program and the user behavior log of the gray stage. According to the offline and gray levels of the application program and the overall test coverage rate, the test result of the whole test process of the application program before formal release can be evaluated. Thus, coverage closer to the business description than code coverage can be obtained. Further, since the behavior log can count the state of the core business of the application program, the behavior log is used for calculating the test coverage rate, and the cost for counting the test coverage rate is low.
In one possible implementation, as shown in fig. 2, step S100 may include:
s110, counting corresponding test coverage point sets of each offline test according to test behavior logs of each offline test of an online offline stage of an application program; the elements of the test coverage point set represent the operational behavior of the application being tested in a single offline test.
In this embodiment, the application may perform multiple offline tests at an offline stage, each of which may include one or more test behavior logs. The test behavior log may include information of time, operation behavior, operation result, and the like of the test application. Thus, all operational behaviors in the offline test can be counted based on the test behavior log of each offline test and assembled. For example, the offline phase includes m tests, and the set of test coverage points included in the offline phase may be expressed as { α } 1 ,α 2 ,…,α m }。
S120, merging elements contained in the test coverage point sets of each off-line test to obtain an off-line coverage point set of the application program.
For example, the set of test coverage points involved in the offline stage may be expressed as { α } 1 ,α 2 ,…,α m The set of offline coverage points of the application may be represented as
Assuming that a certain application program offline stage is tested 3 times in total, a test coverage point set alpha corresponding to the first offline stage 1 The coverage points are { P1, P3, P5}, and the test coverage point set alpha corresponding to the second offline stage 2 The coverage points are { P2, P3, P5}, and the third offline stage corresponds to the test coverage point set alpha 3 Including coverage points { P1, P2, P3}, the application's offline coverage point set is α 1 、α 2 、α 3 Is { P1, P2, P3, P5}.
S130, according to the offline coverage point set and the embedded point complete set of the application program, acquiring the offline test coverage rate of the application program, wherein the offline test coverage rate is the ratio of the number of elements of the offline coverage point set to the number of elements of the embedded point complete set, and the elements of the embedded point complete set represent all the operable behaviors of the application program.
Illustratively, the buried point corpus may be denoted as a, and the offline test coverage may be denoted as:
if connected with the example
The number of elements is 4, the number of elements in the buried point corpus is 10, and P (A) is 40% when { P1, P2, P3, P5}.
In one possible implementation, as shown in fig. 3, step S200 may include:
s210, counting a corresponding usage coverage point set of each user according to a user behavior log of each user using an application program in a gray level stage; the elements of the set of overlay points are used to represent the operational behavior of a single user running an application.
In this embodiment, the application may be provided to a portion of the users for use in the gray scale phase, such as a professional fan. During the process of using the application program, the application program can upload the user behavior log to the server in real time or according to the preset time. For any user, the user behavior log of its use application may include one or more. The user behavior log may record information such as a time when the user uses the application, an operation behavior, and an operation result. Thus, all operational behaviors of each user using the application program can be counted based on the user behavior log of the user, and a set can be composed. For example: the gray phase comprises n users, and the set of usage coverage points comprised by the gray phase can be expressed as { beta } 1 ,β 2 ,…,β n }。
S220, merging elements contained in the using coverage point set of each user to obtain a gray coverage point set of the application program.
Illustratively, the set of usage coverage points involved in the gray phase may be expressed as { β 1 ,β 2 ,…,β n The gray coverage point set of the application can be expressed as
Assuming that the number of test users for a gray level of an application program is 3, the first user corresponds to the coverage point set beta 1 The coverage points are { P1, P4, P5}, and the coverage point set beta is used by the second user 2 The coverage points are { P1, P3, P5}, and the coverage point set beta is used corresponding to the third user 3 The coverage points are { P1, P2, P3}, the gray coverage point set of the application is beta 1 、β 2 、β 3 Is { P1, P2, P3, P4, P5}.
S230, according to the gray scale coverage point set and the embedded point total set of the application program, acquiring gray scale test coverage rate of the application program, wherein the gray scale test coverage rate is the ratio of the number of elements of the gray scale coverage point set to the number of elements of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program.
Illustratively, the buried point corpus may be denoted as a, and the gray test coverage may be denoted as:
if connected with the example
The number of elements is 5, the number of elements in the buried point corpus is 10, and P (B) is 50% when { P1, P2, P3, P4, P5}.
In one possible implementation, as shown in fig. 4, step S300 includes:
And S310, merging elements in the offline coverage point set and the gray coverage point set to obtain an overall coverage point set of the application program.
Illustratively, the set of online underlying coverage points is represented as
The gray scale overlay point set is denoted +.>
The overall set of coverage points for an application can be expressed as: />
Connect with example, if->
Including { P1, P2, P3, P5}, -A }>
Includes { P1, P2, P3, P4, P5}, then ∈>
Including { P1, P2, P3, P4, P5}.
S320, obtaining the overall test coverage rate of the application program according to the overall coverage point set and the embedded point complete set of the application program, wherein the overall test coverage rate is the ratio between the element number of the overall coverage point set and the element number of the embedded point complete set, and the elements of the embedded point complete set represent all the operable behaviors of the application program.
Illustratively, the buried point corpus may be denoted as a and the overall test coverage may be denoted as:
if connected with the example
The number of elements is 5, the number of elements of the buried point corpus is 10, and P (U) is 50% including { P1, P2, P3, P4, P5}.
In one possible implementation, in calculating the overall test coverage, the overall coverage point set of the application may be used to screen out the operation behaviors that are not covered by the test from the operation behaviors included in the application. In the present embodiment, it is possible to check, for an operation behavior that is not covered by the test, why the operation behavior is not covered by the test, whether the case where it is not covered by the test is normal, and the like.
Fig. 5 is a schematic diagram of an application example of coverage of an application program test according to an embodiment of the present invention. In the present application example, the test procedure of the application program includes an off-line phase and a gradation phase. The set of buried points of the application program is exemplified by square set in fig. 5, the test coverage point set in the off-line stage (illustrated by circle set covered by off-line test in fig. 5) includes 2, the usage coverage point set in the gray scale stage (illustrated by circle set covered by gray scale test in fig. 5) includes 3, and the distribution situation of each point set can be as illustrated in fig. 5.
In the embodiment of the present invention, after calculating the under-line test coverage, the gradation test coverage, and the overall test coverage, the evaluation of any one of the following may be performed, respectively:
first, according to the off-line test coverage, evaluating the test result of the off-line stage. For example, when the offline test coverage is greater than the offline test coverage threshold, it may be assumed that the test mode adopted in the offline stage is appropriate and the number of tests is reasonable. When the offline test coverage is smaller than the offline test coverage threshold, it may be assumed that the test mode adopted in the offline stage is to be adjusted and the number of tests needs to be increased. When the offline test coverage is smaller than the offline test coverage threshold, the operation behaviors which are not tested in the application program can be screened from the offline coverage point set counted in the offline test coverage counting process. For the operation behavior which is not covered by the test, it is possible to check why the operation behavior is not covered by the test, whether the case where it is not covered by the test is normal, and the like.
And secondly, evaluating a test result of the gray level stage according to the gray level test coverage rate. For example, when the test coverage of the gray level stage is greater than the gray level test coverage threshold, it may be assumed that the test mode adopted in the gray level stage is proper and the number of tests is reasonable. When the gray test coverage is smaller than the gray test coverage threshold, it may be assumed that the test mode adopted in the gray stage is to be adjusted and the user of the test needs to be increased. When the gray test coverage rate is smaller than the gray test coverage threshold value, the operation behaviors which are not used by the user in the application program can be screened out through the gray test coverage rate counted in the statistical process of the gray test coverage rate. For an operation behavior which has not been used by the user, it is possible to check why the user behavior is not covered by the user, whether or not the case where it is not covered by the user is normal, and the like.
Thirdly, evaluating the test result of the whole stage according to the off-line test coverage, the gray scale test coverage and the whole test coverage. For example, when the overall test coverage is greater than the overall test coverage threshold, it can be assumed that the overall test includes two phases (off-line phase and gray-scale phase) in an appropriate test manner, and the number of tests is reasonable. And when the overall test coverage is smaller than the overall test coverage threshold, comparing the magnitude of the offline test coverage with the magnitude of the offline test coverage, and comparing the magnitude of the gray test coverage with the magnitude of the gray test coverage. And when the offline test coverage rate is smaller than the offline test coverage threshold, adjusting the test mode and the test times of the offline stage. And when the gray test coverage rate is smaller than the gray test coverage threshold value, adjusting the test mode of the offline stage and the number of users to be tested. And after the adjustment, the test is performed again. And re-counting the offline test coverage, the gray scale test coverage and the overall test coverage according to the behavior log of the test process so as to evaluate the adjusted test result.
As shown in fig. 6, an embodiment of the present invention provides an evaluation apparatus for application testing, including:
an offline coverage statistics module 100, configured to count an offline coverage point set and an offline test coverage of an application according to a test behavior log of an online stage of the application;
the gray coverage statistics module 200 is configured to count a gray coverage point set and a gray test coverage rate of the application program according to a user behavior log of the application program in a gray stage;
the overall coverage statistics module 300 is configured to count overall test coverage according to the offline coverage point set and the gray coverage point set; and
and the test result evaluation module 400 is configured to evaluate a test result of the application program according to the offline test coverage, the gray scale test coverage and the overall test coverage.
In one possible implementation, the offline coverage statistics module 100 includes:
the test coverage point statistics unit is used for counting a test coverage point set corresponding to each offline test according to the test behavior log of each offline test of the online offline stage of the application program; the elements of the test coverage point set represent the operation behavior of the application program tested in a single offline test;
The off-line coverage statistics unit is used for merging elements contained in the test coverage point sets of each off-line test to obtain an off-line coverage point set of the application program;
the off-line coverage rate calculation unit is used for obtaining the off-line test coverage rate of the application program according to the off-line coverage point set and the buried point total set of the application program, wherein the off-line test coverage rate is the ratio of the number of elements of the off-line coverage point set to the number of elements of the buried point total set, and the elements of the buried point total set represent all the operable behaviors of the application program.
In one possible implementation, the gray coverage statistics module 200 includes:
the user coverage point statistics unit is used for counting a corresponding usage coverage point set of each user according to the user behavior log of each user using the application program in the gray level stage; the elements of the set of usage coverage points represent the operational behavior of a single user running the application;
the gray coverage statistics unit is used for merging elements contained in the coverage point set of each user to obtain a gray coverage point set of the application program; and
the gray coverage rate calculation unit is used for obtaining gray test coverage rate of the application program according to the gray coverage point set and the embedded point total set of the application program, wherein the gray test coverage rate is a ratio of the number of elements of the gray coverage point set to the number of elements of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program.
In one possible implementation, the overall coverage statistics module 300 includes:
the integral coverage point statistics unit is used for merging the elements in the offline coverage point set and the gray coverage point set to obtain an integral coverage point set of the application program;
the whole coverage rate calculation unit is used for obtaining the whole test coverage rate of the application program according to the whole coverage point set and the embedded point whole set of the application program, wherein the whole test coverage rate is the ratio of the element number of the whole coverage point set to the element number of the embedded point whole set, and the elements of the embedded point whole set represent all the operable behaviors of the application program.
In one possible implementation, as shown in fig. 7, the apparatus includes:
and the uncovered point screening module 500 is configured to screen out operation behaviors of an uncovered test from operation behaviors included in the application program according to the whole covered point set of the application program.
In one possible implementation, the offline stage includes a stage of testing the application before the application is not released for use by a user; and the gray scale stage comprises a stage that the application program is opened for a part of users to use.
The functions of the device can be realized by hardware, and also can be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the functions described above.
In one possible design, the evaluation structure of the application test includes a processor and a memory, where the memory is configured to execute the evaluation program of the application test in the first aspect, and the processor is configured to execute the program stored in the memory. The evaluation means of the application test may further comprise a communication interface for communication of the evaluation means of the application test with other devices or with a communication network.
The embodiment of the invention also provides a terminal device, as shown in fig. 8, which comprises: memory 21 and processor 22, and memory 21 stores a computer program that is executable on processor 22. The processor 22 when executing the computer program implements the survey data based decision method in the above embodiments. The number of memories 21 and processors 22 may be one or more.
The apparatus further comprises:
a communication interface 23 for communication between the processor 22 and an external device.
The memory 21 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 21, the processor 22 and the communication interface 23 are implemented independently, the memory 21, the processor 22 and the communication interface 23 may be connected to each other and perform communication with each other through a bus. The bus may be an industry standard architecture (ISA, industry Standard Architecture) bus, a peripheral component interconnect (PCI, peripheral Component) bus, or an extended industry standard architecture (EISA, extended Industry Standard Component) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 8, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 21, the processor 22 and the communication interface 23 are integrated on a chip, the memory 21, the processor 22 and the communication interface 23 may communicate with each other through internal interfaces.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The computer readable medium of the embodiments of the present invention may be a computer readable signal medium or a computer readable storage medium or any combination of the two. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include at least the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium upon which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
In an embodiment of the invention, the computer readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with computer readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, input method, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, radio Frequency (RF), and the like, or any suitable combination of the foregoing.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or part of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, where the program when executed includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product. The storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that various modifications and substitutions are possible within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (6)
1. A method for evaluating application testing, comprising:
according to a test behavior log of an application program in an online stage, counting an online coverage point set and an online test coverage rate of the application program;
according to the user behavior log of the application program in the gray level stage, counting a gray level coverage point set and gray level test coverage rate of the application program;
according to the off-line coverage point set and the gray coverage point set, counting the overall test coverage rate;
evaluating a test result of the application program according to the off-line test coverage, the gray scale test coverage and the overall test coverage;
the method is characterized in that the statistics of the offline coverage point set and the offline test coverage rate of the application program according to the test behavior log of the online lower stage of the application program comprises the following steps:
According to the test behavior logs of each off-line test of the on-line stage of the application program, counting a corresponding test coverage point set of each off-line test; the elements of the test coverage point set represent the operation behavior of the application program tested in a single offline test;
the elements contained in the test coverage point sets of each off-line test are combined to obtain an off-line coverage point set of the application program;
obtaining an offline test coverage rate of the application program according to the offline coverage point set and the buried point total set of the application program, wherein the offline test coverage rate is a ratio of the number of elements of the offline coverage point set to the number of elements of the buried point total set, and the elements of the buried point total set represent all the operable behaviors of the application program;
and counting a gray coverage point set and a gray test coverage rate of the application program according to the user behavior log of the application program in the gray stage, wherein the method comprises the following steps:
according to the user behavior log of each user using the application program in the gray level stage, counting a corresponding use coverage point set of each user; the elements of the set of usage coverage points represent the operational behavior of a single user running the application;
The elements contained in the using coverage point set of each user are combined to obtain a gray coverage point set of the application program;
obtaining gray test coverage rate of the application program according to the gray coverage point set and the buried point total set of the application program, wherein the gray test coverage rate is the ratio of the number of elements of the gray coverage point set to the number of elements of the buried point total set, and the elements of the buried point total set represent all the operable behaviors of the application program;
and according to the off-line coverage point set and the gray coverage point set, counting the overall test coverage rate, wherein the method comprises the following steps of:
the elements in the offline coverage point set and the gray coverage point set are combined to obtain an overall coverage point set of the application program;
and obtaining the overall test coverage rate of the application program according to the overall coverage point set and the embedded point total set of the application program, wherein the overall test coverage rate is the ratio between the element number of the overall coverage point set and the element number of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program.
2. The method of evaluating an application test of claim 1, wherein the method further comprises:
And screening out operation behaviors which are not covered by the test from operation behaviors contained in the application program according to the whole covered point set of the application program.
3. An evaluation method of application testing according to claim 1 or 2, wherein the offline phase comprises a phase of testing the application before it is not released for use by a user; and the gray scale stage comprises a stage that the application program is opened for a part of users to use.
4. An evaluation device for application program testing, comprising:
the off-line coverage statistics module is used for counting an off-line coverage point set and an off-line test coverage rate of the application program according to a test behavior log of the application program at an off-line stage;
the gray coverage statistics module is used for counting gray coverage point sets and gray test coverage rate of the application program according to the user behavior log of the application program in the gray stage;
the integral coverage statistics module is used for counting integral test coverage according to the off-line coverage point set and the gray coverage point set; and
the test result evaluation module is used for evaluating the test result of the application program according to the off-line test coverage rate, the gray level test coverage rate and the overall test coverage rate;
The off-line coverage statistics module is characterized by comprising:
the test coverage point statistics unit is used for counting a test coverage point set corresponding to each offline test according to the test behavior log of each offline test of the online offline stage of the application program; the elements of the test coverage point set represent the operation behavior of the application program tested in a single offline test;
the off-line coverage statistics unit is used for merging elements contained in the test coverage point sets of each off-line test to obtain an off-line coverage point set of the application program;
an offline coverage calculating unit, configured to obtain an offline test coverage of the application according to the offline coverage point set and a buried point corpus of the application, where the offline test coverage is a ratio of the number of elements of the offline coverage point set to the number of elements of the buried point corpus, and the elements of the buried point corpus represent all operational behaviors of the application;
the gray coverage statistics module is characterized by comprising:
the user coverage point statistics unit is used for counting a corresponding usage coverage point set of each user according to the user behavior log of each user using the application program in the gray level stage; the elements of the set of usage coverage points represent the operational behavior of a single user running the application;
The gray coverage statistics unit is used for merging elements contained in the coverage point set of each user to obtain a gray coverage point set of the application program; and
the gray coverage rate calculation unit is used for obtaining gray test coverage rate of the application program according to the gray coverage point set and the embedded point total set of the application program, wherein the gray test coverage rate is the ratio of the number of elements of the gray coverage point set to the number of elements of the embedded point total set, and the elements of the embedded point total set represent all the operable behaviors of the application program;
the method is characterized in that the integral coverage statistics module comprises the following steps:
the integral coverage point statistics unit is used for merging the elements in the offline coverage point set and the gray coverage point set to obtain an integral coverage point set of the application program;
the whole coverage rate calculation unit is used for obtaining the whole test coverage rate of the application program according to the whole coverage point set and the embedded point whole set of the application program, wherein the whole test coverage rate is the ratio of the element number of the whole coverage point set to the element number of the embedded point whole set, and the elements of the embedded point whole set represent all the operable behaviors of the application program.
5. An evaluation terminal device for implementing application testing, the terminal device comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of evaluation of application testing of any of claims 1-3.
6. A computer-readable storage medium storing a computer program, characterized in that the program, when executed by a processor, implements an evaluation method of an application test according to any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810581509.8A CN110580214B (en) | 2018-06-07 | 2018-06-07 | Evaluation method and device for application program test, storage medium and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810581509.8A CN110580214B (en) | 2018-06-07 | 2018-06-07 | Evaluation method and device for application program test, storage medium and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110580214A CN110580214A (en) | 2019-12-17 |
| CN110580214B true CN110580214B (en) | 2023-05-05 |
Family
ID=68808865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810581509.8A Active CN110580214B (en) | 2018-06-07 | 2018-06-07 | Evaluation method and device for application program test, storage medium and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110580214B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111831545A (en) * | 2020-01-20 | 2020-10-27 | 北京嘀嘀无限科技发展有限公司 | Test case generation method, test case generation device, computer device, and storage medium |
| CN112527611A (en) * | 2020-09-24 | 2021-03-19 | 上海趣蕴网络科技有限公司 | Product health degree assessment method and system |
| CN113254335B (en) * | 2021-05-20 | 2024-04-16 | 北京达佳互联信息技术有限公司 | Test data processing method and device, server and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706751A (en) * | 2009-11-23 | 2010-05-12 | 中兴通讯股份有限公司 | Method and system for calculating coverage rate of service functions of software |
| CN103246602A (en) * | 2012-02-14 | 2013-08-14 | 阿里巴巴集团控股有限公司 | Code coverage rate confirming method, code coverage rate confirming system, code coverage rate detecting method and code coverage rate detecting system |
| CN105404585A (en) * | 2015-12-01 | 2016-03-16 | 腾讯科技（深圳）有限公司 | Method and apparatus for acquiring code coverage rate |
| CN106294151A (en) * | 2016-08-09 | 2017-01-04 | 合智能科技（深圳）有限公司 | Daily record method of testing and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8381184B2 (en) * | 2008-09-02 | 2013-02-19 | International Business Machines Corporation | Dynamic test coverage |
| CN103095518B (en) * | 2011-11-08 | 2016-08-10 | 阿里巴巴集团控股有限公司 | Coverage rate test processing method, device, coverage rate test server and system |
| US9208062B1 (en) * | 2012-08-14 | 2015-12-08 | Amazon Technologies, Inc. | Promotion determination based on aggregated code coverage metrics |
| US8819634B2 (en) * | 2012-09-28 | 2014-08-26 | Sap Ag | System and method to validate test cases |
| CN105224460A (en) * | 2015-10-23 | 2016-01-06 | 浪潮电子信息产业股份有限公司 | Method for counting test coverage rate based on plug-in technology |
| CN107590066A (en) * | 2016-07-08 | 2018-01-16 | 广州市动景计算机科技有限公司 | Method of testing, equipment and the programmable device of application program |
| US11294799B2 (en) * | 2016-09-16 | 2022-04-05 | Oracle International Corporation | Plugin for multi-module code coverage in web applications |
-
2018
- 2018-06-07 CN CN201810581509.8A patent/CN110580214B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706751A (en) * | 2009-11-23 | 2010-05-12 | 中兴通讯股份有限公司 | Method and system for calculating coverage rate of service functions of software |
| CN103246602A (en) * | 2012-02-14 | 2013-08-14 | 阿里巴巴集团控股有限公司 | Code coverage rate confirming method, code coverage rate confirming system, code coverage rate detecting method and code coverage rate detecting system |
| CN105404585A (en) * | 2015-12-01 | 2016-03-16 | 腾讯科技（深圳）有限公司 | Method and apparatus for acquiring code coverage rate |
| CN106294151A (en) * | 2016-08-09 | 2017-01-04 | 合智能科技（深圳）有限公司 | Daily record method of testing and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110580214A (en) | 2019-12-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110580214B (en) | Evaluation method and device for application program test, storage medium and terminal equipment | |
| US6134675A (en) | Method of testing multi-core processors and multi-core processor testing device | |
| CN112199293A (en) | Software quality evaluation method and device, terminal equipment and storage medium | |
| US20080276129A1 (en) | Software tracing | |
| DE112005001371T5 (en) | Method and apparatus for coupling between a test system and an embedded memory for a test mode setting process | |
| DE112012005320T5 (en) | Multicore processor with internally integrated decision-based self-test | |
| US11231854B2 (en) | Methods and apparatus for estimating the wear of a non-volatile memory | |
| CN112052160B (en) | Code use case acquisition method and device, electronic equipment and medium | |
| CN107832208A (en) | The automatic test approach and terminal device of software | |
| CN112333246B (en) | ABtest experiment method and device, intelligent terminal and storage medium | |
| US20160217021A1 (en) | Hardware signal logging in embedded block random access memory | |
| US9646252B2 (en) | Template clauses based SAT techniques | |
| CN108874676A (en) | Method and device for distributing test resources | |
| DE102012221253A1 (en) | Floating point event counter with automatic presentation | |
| US6675323B2 (en) | Incremental fault dictionary | |
| CN114355171B (en) | Reconfigurable chip performance bottleneck detection platform, detection method and electronic equipment | |
| US10051116B2 (en) | Line diagnosis method and apparatus | |
| US7831879B2 (en) | Generating test coverage bin based on simulation result | |
| CN111931930A (en) | Model pruning method and device and electronic equipment | |
| CN112463577B (en) | Sample data processing method and device and electronic equipment | |
| CN110928788B (en) | Service verification method and device | |
| CN110018814A (en) | Unique identifier providing method, storage medium, electronic equipment and system | |
| CN118335171A (en) | Solid state disk testing method and device, electronic equipment and medium | |
| CN111582347B (en) | Fault diagnosis method and device | |
| CN114546815A (en) | Software reliability evaluation method based on software defect number and reliability model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |