CN111639031A - Coverage evaluation method and device based on test data - Google Patents
Coverage evaluation method and device based on test data Download PDFInfo
- Publication number
- CN111639031A CN111639031A CN202010482338.0A CN202010482338A CN111639031A CN 111639031 A CN111639031 A CN 111639031A CN 202010482338 A CN202010482338 A CN 202010482338A CN 111639031 A CN111639031 A CN 111639031A
- Authority
- CN
- China
- Prior art keywords
- test
- coverage
- value
- sub
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides a coverage evaluation method based on test data, which comprises the following steps: obtaining a plurality of test sub-modules included in a tested system; acquiring an actual test value and an expected test value of the test process data of each test submodule, and calculating a subtest coverage value of each test submodule; calculating the total test coverage value of the tested system according to each sub-test coverage value; and evaluating the test process of the tested system according to the sub-test coverage value and the total test coverage value. The invention evaluates the test process of the tested system by comparing each sub-test coverage value with the total test coverage value, and is convenient for the test personnel to adjust the test work, thereby effectively solving the problems that the test efficiency is not high and the test plan cannot be adjusted scientifically and effectively in real time because the test coverage degree is judged according to the experience of the test management personnel in the prior art.
Description
Technical Field
The invention relates to the technical field of computer testing, in particular to a coverage evaluation method and device based on test data.
Background
Software testing is an indispensable flow in a software development process, and mainly relates to test planning, including factors such as personnel, environment, test period and the like. Once the test is started, the test manager cannot adjust the test plan in real time according to the execution condition of the system under test.
Generally, after the test plan is completely executed, the coverage degree of the test can be simply analyzed, that is, the test condition is determined according to the experience of the test manager, so that the test efficiency is low, and the test plan cannot be scientifically and effectively adjusted in real time.
Therefore, a method for scientifically and objectively evaluating the coverage degree of a test is urgently needed, and adjustment of test work in real time is facilitated.
It is noted that this section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Disclosure of Invention
The embodiment of the invention provides a coverage evaluation method and device based on test data, and aims to solve the problems that in the prior art, the coverage degree of a test is judged according to the experience of test managers, so that the test efficiency is low, and the test plan cannot be scientifically and effectively adjusted in real time.
In a first aspect, an embodiment of the present invention provides a coverage evaluation method based on test data, where the method includes:
obtaining a plurality of test sub-modules included in a tested system;
acquiring an actual test value and an expected test value of the test process data of each test sub-module, and calculating a sub-test coverage value of each test sub-module according to the actual test value and the expected test value of the test process data;
calculating the total test coverage value of the tested system according to each sub-test coverage value;
and evaluating the test process of the tested system according to the sub-test coverage value and the total test coverage value.
As a preferred mode of the first aspect of the present invention, the obtaining an actual test value and an expected test value of test process data of each of the test sub-modules, and calculating the sub-test coverage value of each of the test sub-modules according to the actual test value and the expected test value of the test process data includes:
acquiring actual test values and expected test values of various test process data of each test submodule;
calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and through actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) (actual test value of first test procedure data/expected test value of first test procedure data + actual test value of second test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, K is the number of test process data selected from the test sub-modules, coverage (M)i) For testing sub-module MiSub-test coverage value of (a).
As a preferred mode of the first aspect of the present invention, the test procedure data at least includes test defect data, test time data, test log exception record data, and test case data.
As a preferred aspect of the first aspect of the present invention, the calculating a total test coverage value of the system under test according to each of the sub-test coverage values includes:
according to an overall coverage evaluation formula, calculating a total test coverage value of the tested system through each sub-test coverage value; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
In a second aspect, an embodiment of the present invention provides a coverage evaluation apparatus based on test data, including:
the module obtaining unit is used for obtaining a plurality of test sub-modules included in the tested system;
the first calculation unit is used for acquiring an actual test value and an expected test value of the test process data of each test sub-module and calculating a sub-test coverage value of each test sub-module according to the actual test value and the expected test value of the test process data;
the second calculation unit is used for calculating the total test coverage value of the tested system according to each sub-test coverage value;
and the test evaluation unit is used for evaluating the test process of the tested system according to each sub-test coverage value and the total test coverage value.
As a preferred mode of the second aspect of the present invention, the first calculating unit is specifically configured to:
acquiring actual test values and expected test values of various test process data of each test submodule;
calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and through actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) (actual test value of first test procedure data/expected test value of first test procedure data + actual test value of second test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, K is the number of test process data selected from the test sub-modules, coverage (M)i) For testing sub-module MiSub-test coverage value of (a).
As a preferred mode of the second aspect of the present invention, the test procedure data at least includes test defect data, test time data, test log exception record data, and test case data.
As a preferable mode of the second aspect of the present invention, the second calculating unit is specifically configured to:
according to an overall coverage evaluation formula, calculating a total test coverage value of the tested system through each sub-test coverage value; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
In a third aspect, an embodiment of the present invention provides a computing device, including a processor and a memory, where the memory has stored therein execution instructions, and the processor reads the execution instructions in the memory for executing the steps of the coverage evaluation method based on test data as described above.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium containing computer-executable instructions for performing the steps of the coverage assessment method based on test data as described above.
According to the coverage evaluation method and device based on the test data, provided by the embodiment of the invention, the actual test data and the corresponding expected test data of each test sub-module are obtained by the tested system in the test process, then the sub-test coverage value of each test sub-module is calculated, the total test coverage value of the tested system is further calculated, and finally the test process of the tested system is evaluated by comparing each sub-test coverage value and the total test coverage value, so that the test worker can conveniently adjust the test work, and the problems that in the prior art, the test efficiency is low and the test planning cannot be scientifically and effectively adjusted in real time due to the fact that the test coverage degree is judged according to the experience of the test manager are effectively solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a coverage evaluation method based on test data according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a coverage evaluation apparatus based on test data according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Referring to fig. 1, an embodiment of the present invention discloses a coverage evaluation method based on test data, which mainly includes the following steps:
101. obtaining a plurality of test sub-modules included in a tested system;
102. acquiring an actual test value and an expected test value of the test process data of each test submodule, and calculating a subtest coverage value of each test submodule according to the actual test value and the expected test value of the test process data;
103. calculating the total test coverage value of the tested system according to each sub-test coverage value;
104. and evaluating the test process of the tested system according to the sub-test coverage value and the total test coverage value.
In the software development process, the software testing is an indispensable process, and mainly relates to test planning, including factors such as personnel, environment, test period and the like. Once the test work of the software is started to be executed, the test manager cannot adjust the test plan in real time according to the execution condition of the tested system.
Generally, after the test plan is completely executed, the coverage degree of the test can be simply analyzed, that is, the test condition is determined according to the experience of the test manager, so that the test efficiency is low, and the test plan cannot be scientifically and effectively adjusted in real time.
Aiming at the problems that the test coverage degree is judged according to the experience of test managers, so that the test efficiency is not high, and the test plan cannot be scientifically and effectively adjusted in real time, a corresponding method capable of scientifically and objectively evaluating the test coverage degree is provided, so that the adjustment of the test work can be conveniently carried out in real time.
In step 101, for a system M to be tested, N test sub-modules M included in the system M are analyzed firstiWherein M isiFor the ith test submodule included in the system under test M, i is 1, 2, 3 … … N.
In this embodiment, the obtaining process of the plurality of test sub-modules included in the system under test is not limited, and a person skilled in the art can obtain the test sub-modules according to actual situations.
In step 102, after obtaining the plurality of test sub-modules included in the system under test in step 101, an actual test value and an expected test value of test process data obtained by each test sub-module in the test process are further obtained. And then, calculating the sub-test coverage value of each test sub-module according to the obtained actual test value and the expected test value of the test process data of each test sub-module.
When the actual test value and the expected test value of the test process data obtained by each test submodule in the test process are obtained, a plurality of different test process data can be selected to calculate the sub-test coverage value of each test submodule according to the actual execution condition of the tested system, so that the calculated sub-test coverage value is more objective and accurate.
In this embodiment, the process of acquiring the actual test value and the expected test value of the test process data of each test sub-module in the test process is not limited, and those skilled in the art can acquire the actual test value and the expected test value according to the actual situation.
Preferably, in one possible implementation, step 102 may be implemented as follows:
1021. and acquiring actual test values and expected test values of various test process data of each test sub-module.
In the step, the actual test values and the expected test values of various different test process data can be selected according to the actual execution condition of the tested system, so that the finally calculated sub-test coverage value is more objective and accurate.
Further preferably, the test procedure data at least includes test defect data, test time data, test log exception record data and test case data.
The types of the optimized test process data basically cover various factors influencing the test result, so that the optimized test process data can comprehensively consider the factors influencing the test result, and finally, the calculated sub-test coverage value is more objective and accurate.
1022. Calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) (actual test value of first test procedure data/expected test value of first test procedure data + actual test value of second test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, and K is the number of test process data selected from the test sub-modules，coverage(Mi) For testing sub-module MiSub-test coverage value of (a).
In the step, after the actual test values and the expected test values of the various test process data of each test submodule are obtained through the steps, the sub-test coverage value of each test submodule is calculated by adopting a coverage evaluation calculation formula.
It should be noted that, when the sub-test coverage value is calculated by using the coverage evaluation calculation formula, if the actual test value of a certain test process data is greater than the corresponding expected test value, the maximum value of the test process data is the expected test value, and therefore, the ratio of the actual test value to the expected test value of the test process data is directly 1; if the expected test value of a certain test process data is 0 and the actual test value is greater than 0, the maximum value of the test process data is 0, so that the ratio of the actual test value to the expected test value of the test process data is directly 0, and if the actual test value is 0, the actual test value of the test process data is equal to the expected test value, and the ratio of the actual test value to the expected test value of the test process data is directly 1.
Illustratively, table 1 shows the obtained test sub-module MiActual test values and expected test values of the plurality of test procedure data as follows:
TABLE 1
In the above table, sub-module M is testediThe selected test process data comprises four types of test defect data, test time data, test log abnormal record data and test case data, and actual test values and expected test values of various data are correspondingly obtained.
If the data in the above table are correspondingly substituted into the coverage evaluation calculation formula according to the above substitution rule, coverage (Mi) (5/8+1+0+8/10)/4 × 100% (% 60), so that the sub-test coverage value of the test sub-module Mi in the system under test M is 60%.
According to the specific example, the sub-test coverage values of the test sub-modules in the tested system can be calculated in sequence, and accordingly the coverage degree of each test sub-module in the test process can be obtained. Specifically, the larger the calculated coverage (Mi) value is, the higher the test coverage degree of the testing sub-module Mi is, and the smaller the calculated coverage (Mi) value is, the lower the test coverage degree of the testing sub-module Mi is.
In step 103, further summarizing and calculating the total test coverage value of the tested system according to the sub-test coverage values of the test sub-modules obtained in step 102.
Preferably, in a possible implementation manner, the total test coverage value of the tested system is calculated through each sub-test coverage value according to the total coverage evaluation formula; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
In the step, the total test coverage value of the tested system is summarized and calculated by adopting a coverage evaluation calculation formula according to the sub-test coverage values of the test sub-modules obtained in the step. The total test coverage value is preferably the average value of all the sub-test coverage values, so that the total test coverage value is more objective and can reflect the overall test coverage condition.
In step 104, the sub-test coverage values of each test sub-module and the total test coverage value of the tested system are respectively calculated through steps 102 and 103, and the test process of the tested system is evaluated, so that the test coverage degree is judged according to the evaluation. A test project responsible person can master the actual situation of the test in real time and scientifically and effectively adjust the test strategy according to the actual situation so as to ensure the test efficiency.
In summary, the coverage evaluation method based on test data provided in the embodiment of the present invention calculates the sub-test coverage value of each test sub-module through the actual test data and the corresponding expected test data of each test sub-module obtained by the tested system during the test process, further calculates the total test coverage value of the tested system, and finally evaluates the test process of the tested system by comparing each sub-test coverage value with the total test coverage value, so as to facilitate the adjustment of the test work by the tester, thereby effectively solving the problems that the test efficiency is not high and the test plan cannot be adjusted scientifically and effectively in real time due to the fact that the test coverage degree is determined according to the experience of the test manager in the prior art.
It should be noted that the above-mentioned embodiments of the method are described as a series of actions for simplicity of description, but those skilled in the art should understand that the present invention is not limited by the described sequence of actions. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 2, based on the same inventive concept, an embodiment of the present invention provides a coverage evaluation apparatus based on test data, which mainly includes:
a module obtaining unit 21, configured to obtain a plurality of test sub-modules included in a system under test;
the first calculating unit 22 is configured to obtain an actual test value and an expected test value of the test process data of each test sub-module, and calculate a sub-test coverage value of each test sub-module according to the actual test value and the expected test value of the test process data;
the second calculating unit 23 is configured to calculate a total test coverage value of the system under test according to each of the sub-test coverage values;
and the test evaluation unit 24 is used for evaluating the test process of the tested system according to each sub-test coverage value and the total test coverage value.
Preferably, the first computing unit 22 is specifically configured to:
acquiring actual test values and expected test values of various test process data of each test submodule;
calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) (actual test value of first test procedure data/expected test value of first test procedure data + actual test value of second test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, K is the number of test process data selected from the test sub-modules, coverage (M)i) For testing sub-module MiSub-test coverage value of (a).
Preferably, the test procedure data at least includes test defect data, test time data, test log exception record data and test case data.
Preferably, the second calculation unit 23 is specifically configured to:
calculating the total test coverage value of the tested system through each sub-test coverage value according to a total coverage evaluation formula; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
In summary, the coverage evaluation apparatus based on test data provided in the embodiment of the present invention calculates the sub-test coverage value of each test sub-module through the actual test data and the corresponding expected test data of each test sub-module obtained by the tested system during the test process, further calculates the total test coverage value of the tested system, and finally evaluates the test process of the tested system by comparing each sub-test coverage value with the total test coverage value, so as to facilitate the adjustment of the test work by the tester, thereby effectively solving the problems that the test efficiency is not high and the test plan cannot be adjusted scientifically and effectively in real time due to the fact that the test coverage degree is determined according to the experience of the test manager in the prior art.
It should be understood that the above coverage evaluation device based on test data includes only units that are logically divided according to the functions implemented by the device, and in practical applications, the above units may be stacked or split. The functions implemented by the coverage evaluation device based on test data according to this embodiment correspond to the coverage evaluation method based on test data according to the foregoing embodiment one-to-one, and for the more detailed processing flow implemented by the device, detailed description has been made in the foregoing method embodiment, and detailed description is not given here.
Referring to fig. 3, the computing device mainly includes a processor 31 and a memory 32, wherein the memory 32 stores execution instructions. The processor 31 reads the execution instructions in the memory 32 for executing the steps described in the above embodiments of the coverage evaluation method based on test data. Alternatively, the processor 31 reads the execution instructions in the memory 32 to implement the functions of the units in the above-mentioned coverage evaluation device embodiment based on test data.
Fig. 3 is a schematic structural diagram of a computing device according to an embodiment of the present invention, as shown in fig. 3, the computing device includes a processor 31, a memory 32, and a transceiver 33; wherein the processor 31, the memory 32 and the transceiver 33 are interconnected by a bus 34.
The memory 32 is used for storing programs; in particular, the program may include program code including computer operating instructions. The memory 32 may include a volatile memory (volatile memory), such as a random-access memory (RAM); the memory 32 may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD); the memory 32 may also comprise a combination of the above types of memories.
The memory 32 stores the following elements, executable modules or data structures, or a subset thereof, or an expanded set thereof:
and (3) operating instructions: including various operational instructions for performing various operations.
Operating the system: including various system programs for implementing various basic services and for handling hardware-based tasks.
The bus 34 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.
The processor 31 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of CPU and NP. But also a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA), a General Array Logic (GAL), or any combination thereof.
Embodiments of the present invention further provide a computer-readable storage medium, which contains computer-executable instructions, where the computer-executable instructions are used to perform the steps described in the above coverage evaluation method embodiments based on test data. Alternatively, the computer executable instructions are used to perform the functions of the units in the above-described coverage assessment apparatus embodiments based on test data.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A method for coverage assessment based on test data, the method comprising:
obtaining a plurality of test sub-modules included in a tested system;
acquiring an actual test value and an expected test value of the test process data of each test sub-module, and calculating a sub-test coverage value of each test sub-module according to the actual test value and the expected test value of the test process data;
calculating the total test coverage value of the tested system according to each sub-test coverage value;
and evaluating the test process of the tested system according to the sub-test coverage value and the total test coverage value.
2. The method of claim 1, wherein obtaining the actual test value and the expected test value of the test process data of each of the test sub-modules, and calculating the sub-test coverage value of each of the test sub-modules according to the actual test value and the expected test value of the test process data comprises:
acquiring actual test values and expected test values of various test process data of each test submodule;
calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and through actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) First test data of the first type/expected test data of the first type + second test data of the second typeActual test value of test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, K is the number of test process data selected from the test sub-modules, coverage (M)i) For testing sub-module MiSub-test coverage value of (a).
3. The method of claim 2, wherein the test procedure data includes at least test defect data, test time data, test log exception record data, and test case data.
4. The method of claim 2, wherein calculating the total test coverage value for the system under test from each of the sub-test coverage values comprises:
according to an overall coverage evaluation formula, calculating a total test coverage value of the tested system through each sub-test coverage value; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
5. A coverage assessment apparatus based on test data, the apparatus comprising:
the module obtaining unit is used for obtaining a plurality of test sub-modules included in the tested system;
the first calculation unit is used for acquiring an actual test value and an expected test value of the test process data of each test sub-module and calculating a sub-test coverage value of each test sub-module according to the actual test value and the expected test value of the test process data;
the second calculation unit is used for calculating the total test coverage value of the tested system according to each sub-test coverage value;
and the test evaluation unit is used for evaluating the test process of the tested system according to each sub-test coverage value and the total test coverage value.
6. The apparatus according to claim 5, wherein the first computing unit is specifically configured to:
acquiring actual test values and expected test values of various test process data of each test submodule;
calculating sub-test coverage values of the test sub-modules according to a coverage evaluation calculation formula and through actual test values and expected test values of various test process data of the test sub-modules; the coverage assessment calculation formula is as follows:
coverage(Mi) (actual test value of first test procedure data/expected test value of first test procedure data + actual test value of second test procedure data/expected test value of second test procedure data + … … + actual test value of kth test procedure data/expected test value of kth test procedure data)/K × 100%,
wherein M isiFor the ith test sub-module included in the tested system M, i is 1, 2, 3 … … N, N is the number of test sub-modules, K is the number of test process data selected from the test sub-modules, coverage (M)i) For testing sub-module MiSub-test coverage value of (a).
7. The apparatus of claim 6, wherein the test procedure data comprises at least test defect data, test time data, test log exception record data, and test case data.
8. The apparatus according to claim 6, wherein the second computing unit is specifically configured to:
according to an overall coverage evaluation formula, calculating a total test coverage value of the tested system through each sub-test coverage value; the overall coverage assessment calculation formula is as follows:
coverage(M)＝(∑N i＝1coverage(Mi))/N，
wherein, coverage (M) is the total test coverage value of the tested system M.
9. A computing device comprising a processor and a memory, wherein the memory has stored therein execution instructions, wherein the processor reads the execution instructions in the memory for performing the steps of the method according to any one of claims 1 to 4.
10. A computer-readable storage medium containing computer-executable instructions for performing the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010482338.0A CN111639031B (en) | 2020-06-01 | 2020-06-01 | Coverage evaluation method and device based on test data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010482338.0A CN111639031B (en) | 2020-06-01 | 2020-06-01 | Coverage evaluation method and device based on test data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111639031A true CN111639031A (en) | 2020-09-08 |
| CN111639031B CN111639031B (en) | 2023-06-30 |
Family
ID=72327027
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010482338.0A Active CN111639031B (en) | 2020-06-01 | 2020-06-01 | Coverage evaluation method and device based on test data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639031B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395200A (en) * | 2020-11-25 | 2021-02-23 | 电信科学技术第十研究所有限公司 | Method and device for determining probability of software reliability operation profile |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140123109A1 (en) * | 2012-10-30 | 2014-05-01 | Oracle International Corporation | System and method for testing compiler implementation using dynamic combinatorial test generation |
| CN105629948A (en) * | 2015-12-24 | 2016-06-01 | 北京新能源汽车股份有限公司 | Test case detection method and device |
| CN105824748A (en) * | 2015-01-22 | 2016-08-03 | 国际商业机器公司 | Determining test case efficiency |
| US20170147474A1 (en) * | 2015-11-24 | 2017-05-25 | International Business Machines Corporation | Software testing coverage |
| CN110209576A (en) * | 2019-05-21 | 2019-09-06 | 深圳壹账通智能科技有限公司 | The calculation method and relevant device of test case coverage rate |
-
2020
- 2020-06-01 CN CN202010482338.0A patent/CN111639031B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140123109A1 (en) * | 2012-10-30 | 2014-05-01 | Oracle International Corporation | System and method for testing compiler implementation using dynamic combinatorial test generation |
| CN105824748A (en) * | 2015-01-22 | 2016-08-03 | 国际商业机器公司 | Determining test case efficiency |
| US20170147474A1 (en) * | 2015-11-24 | 2017-05-25 | International Business Machines Corporation | Software testing coverage |
| CN105629948A (en) * | 2015-12-24 | 2016-06-01 | 北京新能源汽车股份有限公司 | Test case detection method and device |
| CN110209576A (en) * | 2019-05-21 | 2019-09-06 | 深圳壹账通智能科技有限公司 | The calculation method and relevant device of test case coverage rate |
Non-Patent Citations (3)
| Title |
|---|
| 张瑶;白晓颖;张任伟;陆皓;: "一种基于模型的测试充分性评估方法" * |
| 李德明;叶进;: "基于配置项前置约束算法的自动测试框架研究" * |
| 高如莎;: "一种软件单元测试策略" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395200A (en) * | 2020-11-25 | 2021-02-23 | 电信科学技术第十研究所有限公司 | Method and device for determining probability of software reliability operation profile |
| CN112395200B (en) * | 2020-11-25 | 2024-04-02 | 电信科学技术第十研究所有限公司 | Method and device for determining probability of software reliability operation profile |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111639031B (en) | 2023-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10678673B2 (en) | Software program fault localization | |
| CN108845933B (en) | Method and device for compiling and reviewing software test cases | |
| EP1136912B1 (en) | Probalistic diagnosis, in particular for embedded and remote applications | |
| US20080222501A1 (en) | Analyzing Test Case Failures | |
| WO2020113526A1 (en) | Chip verification method and device | |
| CN111639031B (en) | Coverage evaluation method and device based on test data | |
| CN104063307A (en) | Software testing method and system | |
| CN111475402A (en) | Program function testing method and related device | |
| US11520689B2 (en) | System and method for automatic program repair using fast-result test cases | |
| US20140281719A1 (en) | Explaining excluding a test from a test suite | |
| CN109902001B (en) | Method for detecting uninitialized variable and terminal equipment | |
| CN114355171B (en) | Reconfigurable chip performance bottleneck detection platform, detection method and electronic equipment | |
| CN109947656A (en) | A kind of testing case management and device | |
| CN113282496B (en) | Automatic interface testing method, device, equipment and storage medium | |
| CN113127324B (en) | Automatic test report generation method and device, computer equipment and storage medium | |
| CN111930630A (en) | Big data test case generation method and device based on data flow | |
| CN113791980A (en) | Test case conversion analysis method, device, equipment and storage medium | |
| CN112131108B (en) | Feature attribute-based test strategy adjustment method and device | |
| CN113257329A (en) | Memory fault diagnosis method based on machine learning | |
| CN110221974A (en) | Service platform system self checking method, device, computer equipment and storage medium | |
| US20090228241A1 (en) | System testing method through subsystem performance-based generator | |
| US20090241076A1 (en) | Test-Cases for Functional Verification of System-Level Interconnect | |
| CN116185746A (en) | Fault position determining method and device and electronic equipment | |
| CN117471294A (en) | Chip testing method and device and electronic equipment | |
| Hribar et al. | Implementation of the Software Quality Ranks method in the legacy product development environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |