CN105224460A - A kind of method based on Program instrumentation statistical test coverage rate - Google Patents
A kind of method based on Program instrumentation statistical test coverage rate Download PDFInfo
- Publication number
- CN105224460A CN105224460A CN201510694988.0A CN201510694988A CN105224460A CN 105224460 A CN105224460 A CN 105224460A CN 201510694988 A CN201510694988 A CN 201510694988A CN 105224460 A CN105224460 A CN 105224460A
- Authority
- CN
- China
- Prior art keywords
- test
- program
- coverage rate
- coverage
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.The present invention is by the test coverage of calculating three levels, and three kinds of coverage rates combine, and to a certain degree minimizing concentrations calls the impact on data; According to project testing stage and coverage data, reasonable, judge flexibly to prevent from the validity of testing shift onto each other, ignoring the fact, improve the science of project management.
Description
Technical field
The present invention relates to software test and project management techniques field, be specifically related to a kind of method based on Program instrumentation statistical test coverage rate, mainly for quality in software development process repeatedly, poor controllability, the problems such as accountability difficulty, by inserting detective pole in program code, monitoring this code and whether being performed in certain test, statistical test coverage rate, reduce randomness, the randomness of test, improve the science of project management, thus shorten the project testing time, reduce testing cost.
Background technology
When being engaged in software project management and test and management, we can find in project in the face of concluding a research item time pressure, now software quality repeatedly, the bug degree of convergence not problem such as expected, the often greatest problem that faces of project concluding a research item.The way that project manager takes usually first is that accountability is followed the trail of in bug classification: classified by this bug, new discovery, amendment cause, unmodified etc., to investigate the responsibility of tester and developer, expect that developer revises bug quality and tester tests comprehensive raising, second is that overtime work is overworked.QA Manager, in order to reduce the quantity of new discovery bug, often adopts the way of stuck-module tester.
Undeniable, these ways can obtain certain effect.But the statistic processes of bug classification, lack science, the standard of system, classify for previous test impression according to exploitation and test, this often causes exploitation and tests the phenomenon of shift onto each other.
Sum up the problem being at all test coverage of these contradictions, how to ensure just comprehensively to cover at the test initial stage? this comprehensive for program test of test how is added up after a test as managerial personnel?
At present, software quality tight at project duration repeatedly, the bug degree of convergence as expected in situation, often do not lack science and reliability when project manager sums up reason, lacks data foundation and cogency to this conclusion project other staff.
Summary of the invention
The technical problem to be solved in the present invention is: based on above-mentioned technology, the invention provides a kind of method based on Program instrumentation statistical test coverage rate.
The technical solution adopted in the present invention is:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
Described code condition branch covering rate, to branch into unit in function, inserts probe when each branch performs, and detecting probe information comprises branch's numbering, in coding stage, and record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether judge that this takes turns test is effective test, for the relation of coverage rate and Validity Test, different phase according to test specifically defines, and the phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Described system testing needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Beneficial effect of the present invention is:
The present invention, by the technology of plug-in mounting in code, records test trails; Probe after taking turns test by one exports, and calculate the test coverage of three levels, three kinds of coverage rates combine, and to a certain degree minimizing concentrations calls the impact on data; According to project testing stage and coverage data, reasonable, judge flexibly to prevent from the validity of testing shift onto each other, ignoring the fact, improve the science of project management.
Accompanying drawing explanation
Fig. 1 calculates test coverage method figure for the present invention is based on Program instrumentation.
Embodiment
Below according to Figure of description, in conjunction with embodiment, the present invention is further described:
Embodiment 1:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Embodiment 2:
On the basis of embodiment 1, api coverage rate described in the present embodiment, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Embodiment 3:
On the basis of embodiment 1, function coverage rate described in the present embodiment, in units of function, before each function starts to perform, insert probe, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, and initial value is 0), when program detailed design, record number is the invoked number of times of function of i, counts total A (i), function coverage rate=X(i)/A (i).
Embodiment 4:
On the basis of embodiment 1, code condition branch covering rate described in the present embodiment, to branch into unit in function, insert probe when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Embodiment 5:
On the basis of embodiment 1-4, method described in the present embodiment is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Embodiment 6:
As shown in Figure 1, on the basis of embodiment 5, described in the present embodiment, method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Embodiment 7:
On the basis of embodiment 6, system testing described in the present embodiment needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Above embodiment is only for illustration of the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all equivalent technical schemes also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.
Claims (7)
1. the method based on Program instrumentation statistical test coverage rate, it is characterized in that: described method inserts some probes in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
2. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, performs degree variables X(i), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered depending on this interface and reject the calculating of i interface in total api coverage rate
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)].
3. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
4. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described code condition branch covering rate, to branch into unit in function, probe is inserted when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
5. according to the arbitrary a kind of described method based on Program instrumentation statistical test coverage rate of claim 1-4, it is characterized in that: described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
6. a kind of method based on Program instrumentation statistical test coverage rate according to claim 5, is characterized in that, described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and branch covering rate respectively, the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, whether test is effectively judged.
7. a kind of method based on Program instrumentation statistical test coverage rate according to claim 6, it is characterized in that: described system testing lasting 5 is taken turns above, test coverage relatively between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510694988.0A CN105224460A (en) | 2015-10-23 | 2015-10-23 | A kind of method based on Program instrumentation statistical test coverage rate |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510694988.0A CN105224460A (en) | 2015-10-23 | 2015-10-23 | A kind of method based on Program instrumentation statistical test coverage rate |
Publications (1)
Publication Number | Publication Date |
---|---|
CN105224460A true CN105224460A (en) | 2016-01-06 |
Family
ID=54993445
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510694988.0A Pending CN105224460A (en) | 2015-10-23 | 2015-10-23 | A kind of method based on Program instrumentation statistical test coverage rate |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN105224460A (en) |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106294163A (en) * | 2016-08-12 | 2017-01-04 | 北京航空航天大学 | A kind of code branch covering rate detection method of lightweight |
CN109542789A (en) * | 2018-11-26 | 2019-03-29 | 泰康保险集团股份有限公司 | A kind of code coverage statistical method and device |
WO2019071929A1 (en) * | 2017-10-09 | 2019-04-18 | 武汉斗鱼网络科技有限公司 | Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device |
CN110580214A (en) * | 2018-06-07 | 2019-12-17 | 百度在线网络技术(北京)有限公司 | application program test evaluation method and device, storage medium and terminal equipment |
CN112597064A (en) * | 2021-03-02 | 2021-04-02 | 芯华章科技股份有限公司 | Method for simulating program, electronic device and storage medium |
CN112597007A (en) * | 2020-12-14 | 2021-04-02 | 中国航发控制系统研究所 | Embedded software integration test integrity analysis method |
CN113094289A (en) * | 2021-05-18 | 2021-07-09 | 绿漫科技有限公司 | Method for realizing interface coverage rate statistics through python language |
US11086759B2 (en) | 2018-09-27 | 2021-08-10 | SeaLights Technologies LTD | System and method for probe injection for code coverage |
US11093374B2 (en) | 2016-08-09 | 2021-08-17 | SeaLights Technologies LTD | System and method for continuous testing and delivery of software |
CN113392033A (en) * | 2021-08-17 | 2021-09-14 | 北京安普诺信息技术有限公司 | Method and device for determining passive IAST test API coverage rate |
CN113535559A (en) * | 2021-07-14 | 2021-10-22 | 杭州默安科技有限公司 | Application interface coverage rate statistical method and system |
US11573885B1 (en) | 2019-09-26 | 2023-02-07 | SeaLights Technologies LTD | System and method for test selection according to test impact analytics |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7334161B2 (en) * | 2004-04-30 | 2008-02-19 | Arm Limited | Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus |
CN102103538A (en) * | 2011-02-22 | 2011-06-22 | 南京航空航天大学 | Method for testing palletizing robot control software based on Agent |
CN102419728A (en) * | 2011-11-01 | 2012-04-18 | 北京邮电大学 | Method for determining software test process sufficiency based on coverage rate quantitative indicators |
-
2015
- 2015-10-23 CN CN201510694988.0A patent/CN105224460A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7334161B2 (en) * | 2004-04-30 | 2008-02-19 | Arm Limited | Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus |
CN102103538A (en) * | 2011-02-22 | 2011-06-22 | 南京航空航天大学 | Method for testing palletizing robot control software based on Agent |
CN102419728A (en) * | 2011-11-01 | 2012-04-18 | 北京邮电大学 | Method for determining software test process sufficiency based on coverage rate quantitative indicators |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11093374B2 (en) | 2016-08-09 | 2021-08-17 | SeaLights Technologies LTD | System and method for continuous testing and delivery of software |
US11775416B2 (en) | 2016-08-09 | 2023-10-03 | SeaLights Technologies LTD | System and method for continuous testing and delivery of software |
CN106294163A (en) * | 2016-08-12 | 2017-01-04 | 北京航空航天大学 | A kind of code branch covering rate detection method of lightweight |
WO2019071929A1 (en) * | 2017-10-09 | 2019-04-18 | 武汉斗鱼网络科技有限公司 | Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device |
CN110580214A (en) * | 2018-06-07 | 2019-12-17 | 百度在线网络技术(北京)有限公司 | application program test evaluation method and device, storage medium and terminal equipment |
US11847041B2 (en) | 2018-09-27 | 2023-12-19 | Sealights Technologies Ltd. | System and method for probe injection for code coverage |
US11086759B2 (en) | 2018-09-27 | 2021-08-10 | SeaLights Technologies LTD | System and method for probe injection for code coverage |
CN109542789B (en) * | 2018-11-26 | 2022-03-25 | 泰康保险集团股份有限公司 | Code coverage rate statistical method and device |
CN109542789A (en) * | 2018-11-26 | 2019-03-29 | 泰康保险集团股份有限公司 | A kind of code coverage statistical method and device |
US11573885B1 (en) | 2019-09-26 | 2023-02-07 | SeaLights Technologies LTD | System and method for test selection according to test impact analytics |
CN112597007A (en) * | 2020-12-14 | 2021-04-02 | 中国航发控制系统研究所 | Embedded software integration test integrity analysis method |
CN112597007B (en) * | 2020-12-14 | 2023-10-03 | 中国航发控制系统研究所 | Integrated test integrity analysis method for embedded software |
CN112597064A (en) * | 2021-03-02 | 2021-04-02 | 芯华章科技股份有限公司 | Method for simulating program, electronic device and storage medium |
CN112597064B (en) * | 2021-03-02 | 2021-06-29 | 芯华章科技股份有限公司 | Method for simulating program, electronic device and storage medium |
CN113094289A (en) * | 2021-05-18 | 2021-07-09 | 绿漫科技有限公司 | Method for realizing interface coverage rate statistics through python language |
CN113535559A (en) * | 2021-07-14 | 2021-10-22 | 杭州默安科技有限公司 | Application interface coverage rate statistical method and system |
CN113392033B (en) * | 2021-08-17 | 2022-03-08 | 北京安普诺信息技术有限公司 | Method and device for determining passive IAST test API coverage rate |
CN113392033A (en) * | 2021-08-17 | 2021-09-14 | 北京安普诺信息技术有限公司 | Method and device for determining passive IAST test API coverage rate |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN105224460A (en) | A kind of method based on Program instrumentation statistical test coverage rate | |
EP1899876B1 (en) | System and method for using model analysis to generate directed test vectors | |
US5500941A (en) | Optimum functional test method to determine the quality of a software system embedded in a large electronic system | |
CN107301119B (en) | Method and device for analyzing IT fault root cause by utilizing time sequence correlation | |
CN102682229B (en) | Malicious code behavior detection method based on virtualization technology | |
Tahat et al. | Regression test suite prioritization using system models | |
WO1997002528A1 (en) | Method and system for an architecture based analysis of software quality | |
CN102445941B (en) | Method for automatically determining and analyzing interoperability test results of on-board equipment of train control system | |
CN107478455A (en) | A kind of Censoring reliability test method suitable for Weibull distribution type product | |
CN111756582A (en) | Service chain monitoring method based on NFV log alarm | |
CN102999417B (en) | Automatic test management system and method | |
CN101697144A (en) | Software test method and software test device | |
CN101482596B (en) | Fast fault recognition method for multifactor industrial system | |
CN109858097A (en) | A kind of spacecraft single machine test assessment methods of sampling | |
CN114691521A (en) | Software testing platform based on artificial intelligence | |
CN110059010A (en) | The buffer overflow detection method with fuzz testing is executed based on dynamic symbol | |
CN116136950B (en) | Chip verification method, device, system, electronic equipment and storage medium | |
CN105653455A (en) | Program vulnerability detection method and detection system | |
CN112506802A (en) | Test data management method and system | |
CN116954624A (en) | Compiling method based on software development kit, software development system and server | |
CN109359047B (en) | Software testing sufficiency measuring method based on system characteristic state | |
Savor et al. | Toward automatic detection of software failures | |
CN112798944B (en) | FPGA hardware error attribution analysis method based on online real-time data | |
CN113079061B (en) | Internet of things performance testing method and system | |
CN105574312A (en) | Method and system for engineering change data analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160106 |