CN105224460A - A kind of method based on Program instrumentation statistical test coverage rate - Google Patents

A kind of method based on Program instrumentation statistical test coverage rate Download PDF

Info

Publication number
CN105224460A
CN105224460A CN201510694988.0A CN201510694988A CN105224460A CN 105224460 A CN105224460 A CN 105224460A CN 201510694988 A CN201510694988 A CN 201510694988A CN 105224460 A CN105224460 A CN 105224460A
Authority
CN
China
Prior art keywords
test
program
coverage rate
coverage
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201510694988.0A
Other languages
Chinese (zh)
Inventor
赵霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Electronic Information Industry Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN201510694988.0A priority Critical patent/CN105224460A/en
Publication of CN105224460A publication Critical patent/CN105224460A/en
Pending legal-status Critical Current

Links

Abstract

The invention discloses a kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.The present invention is by the test coverage of calculating three levels, and three kinds of coverage rates combine, and to a certain degree minimizing concentrations calls the impact on data; According to project testing stage and coverage data, reasonable, judge flexibly to prevent from the validity of testing shift onto each other, ignoring the fact, improve the science of project management.

Description

A kind of method based on Program instrumentation statistical test coverage rate
Technical field
The present invention relates to software test and project management techniques field, be specifically related to a kind of method based on Program instrumentation statistical test coverage rate, mainly for quality in software development process repeatedly, poor controllability, the problems such as accountability difficulty, by inserting detective pole in program code, monitoring this code and whether being performed in certain test, statistical test coverage rate, reduce randomness, the randomness of test, improve the science of project management, thus shorten the project testing time, reduce testing cost.
Background technology
When being engaged in software project management and test and management, we can find in project in the face of concluding a research item time pressure, now software quality repeatedly, the bug degree of convergence not problem such as expected, the often greatest problem that faces of project concluding a research item.The way that project manager takes usually first is that accountability is followed the trail of in bug classification: classified by this bug, new discovery, amendment cause, unmodified etc., to investigate the responsibility of tester and developer, expect that developer revises bug quality and tester tests comprehensive raising, second is that overtime work is overworked.QA Manager, in order to reduce the quantity of new discovery bug, often adopts the way of stuck-module tester.
Undeniable, these ways can obtain certain effect.But the statistic processes of bug classification, lack science, the standard of system, classify for previous test impression according to exploitation and test, this often causes exploitation and tests the phenomenon of shift onto each other.
Sum up the problem being at all test coverage of these contradictions, how to ensure just comprehensively to cover at the test initial stage? this comprehensive for program test of test how is added up after a test as managerial personnel?
At present, software quality tight at project duration repeatedly, the bug degree of convergence as expected in situation, often do not lack science and reliability when project manager sums up reason, lacks data foundation and cogency to this conclusion project other staff.
Summary of the invention
The technical problem to be solved in the present invention is: based on above-mentioned technology, the invention provides a kind of method based on Program instrumentation statistical test coverage rate.
The technical solution adopted in the present invention is:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
Described code condition branch covering rate, to branch into unit in function, inserts probe when each branch performs, and detecting probe information comprises branch's numbering, in coding stage, and record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether judge that this takes turns test is effective test, for the relation of coverage rate and Validity Test, different phase according to test specifically defines, and the phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Described system testing needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Beneficial effect of the present invention is:
The present invention, by the technology of plug-in mounting in code, records test trails; Probe after taking turns test by one exports, and calculate the test coverage of three levels, three kinds of coverage rates combine, and to a certain degree minimizing concentrations calls the impact on data; According to project testing stage and coverage data, reasonable, judge flexibly to prevent from the validity of testing shift onto each other, ignoring the fact, improve the science of project management.
Accompanying drawing explanation
Fig. 1 calculates test coverage method figure for the present invention is based on Program instrumentation.
Embodiment
Below according to Figure of description, in conjunction with embodiment, the present invention is further described:
Embodiment 1:
A kind of method based on Program instrumentation statistical test coverage rate, described method inserts some probes (being also called " detection instrument ") in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
Namely method based on the statistical test coverage rate of Program instrumentation inserts dress probe in tested program, when program is performed (tested), probe performs and inputs appointed information, otherwise, the appointed information of probe does not export, and proves that this section of program is not tested in epicycle test and arrives, thus can science, the convictive coverage rate calculating test.
Embodiment 2:
On the basis of embodiment 1, api coverage rate described in the present embodiment, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, perform degree variables X(i) (interface is often performed once this variable and adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered (this method is got rid of this interface based on the randomness of test and concentrated situation about calling by under a few cases) depending on this interface and reject the calculating of i interface in total api coverage rate,
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)] the i interface of (wherein 1 to n in reject api(i) >1).When api coverage rate is on the low side, can by calculating api(i)=X(i)/A (i) is specific to being certain concrete api.
Embodiment 3:
On the basis of embodiment 1, function coverage rate described in the present embodiment, in units of function, before each function starts to perform, insert probe, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, and initial value is 0), when program detailed design, record number is the invoked number of times of function of i, counts total A (i), function coverage rate=X(i)/A (i).
Embodiment 4:
On the basis of embodiment 1, code condition branch covering rate described in the present embodiment, to branch into unit in function, insert probe when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
Embodiment 5:
On the basis of embodiment 1-4, method described in the present embodiment is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
Embodiment 6:
As shown in Figure 1, on the basis of embodiment 5, described in the present embodiment, method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and the branch covering rate (situation of comprehensive three kinds of coverage rates respectively, situation about calling is concentrated) under can avoiding a few cases to a certain extent, whether the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, effectively judge test.
At the test initial stage, the function of project only completes 50%, according to sixteen principles, if three kinds of test coverages are all lower than 48%, then consider the situation that test coverage is low, again according to the branch covering rate of single interface coverage rate, single function coverage rate and single function, go investigation to be confirmed to be which code and do not cover; Arrived the project later stage, program function completes substantially, then three kinds of test coverages will reach more than 80% even higher substantially, could judge that this time test effectively.
Embodiment 7:
On the basis of embodiment 6, system testing described in the present embodiment needs lasting 5 to take turns even higher, compares the test coverage between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
Above embodiment is only for illustration of the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; without departing from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all equivalent technical schemes also belong to category of the present invention, and scope of patent protection of the present invention should be defined by the claims.

Claims (7)

1. the method based on Program instrumentation statistical test coverage rate, it is characterized in that: described method inserts some probes in a program on the basis ensureing the original logic integrality of tested program, by the execution of probe and the characteristic run of program of dishing out, by the analysis to these data, the control flow check of acquisition program and traffic flow information, and then obtain multidate information test coverage, thus realize test purpose; Described test coverage adopts level coverage rate, is three levels: api coverage rate, function coverage rate, code condition branch covering rate, the coverage rate of three levels is from coarse to fine, interrelated according to program granularity division.
2. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described api coverage rate, in units of routine interface, probe is inserted before each interface performs, detecting probe information comprises interface index, performs degree variables X(i), when program detailed design, record number is the invoked number of times of interface of i, count total A (i), api(i)=X(i)/A (i), as api(i) >1 time, be completely covered depending on this interface and reject the calculating of i interface in total api coverage rate
Total api coverage rate=[X(1)+X(2)+... + X (n-1)+X (n)]/[A (1)+A (2)+... + A (n-1)+A (n)].
3. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described function coverage rate, in units of function, probe was inserted before each function starts to perform, detecting probe information comprises function numbering, performs number of times X(i) (function often called once this variable adds 1, initial value is 0), when program detailed design, record number is the invoked number of times of function of i, count total A (i), function coverage rate=X(i)/A (i).
4. a kind of method based on Program instrumentation statistical test coverage rate according to claim 1, it is characterized in that: described code condition branch covering rate, to branch into unit in function, probe is inserted when each branch performs, detecting probe information comprises branch's numbering, in coding stage, record numbers of branches, the numbers of branches that the numbering of branch covering rate=executions is total/total.
5. according to the arbitrary a kind of described method based on Program instrumentation statistical test coverage rate of claim 1-4, it is characterized in that: described method is when calculation procedure test coverage, the coverage rate of these three kinds of levels comprehensive, whether be effective test, for the relation of coverage rate and Validity Test, the different phase according to test specifically defines if judging that this takes turns test, phase after a test, when program function is tending towards improving, three kinds of coverage rates will reach more than 90% substantially, and test is just calculated effectively.
6. a kind of method based on Program instrumentation statistical test coverage rate according to claim 5, is characterized in that, described method operation steps is as follows:
1) project detailed design phase, according to detailed design figure, to api numbering, and records api by the quantity of called situation; To function numbering, and record the invoked situation quantity of function;
2) in coding stage, number to the branch of function;
3) in coding stage, start, function starts and the branch of function starts at api interface, insert probe: System.out.println statement, input information, to the file of specifying, includes but not limited to numbering and performs number of times;
4) after test terminates, calculate api coverage rate, function coverage rate and branch covering rate respectively, the performance of the current item-function of comprehensive consideration and three kinds of coverage rates, whether test is effectively judged.
7. a kind of method based on Program instrumentation statistical test coverage rate according to claim 6, it is characterized in that: described system testing lasting 5 is taken turns above, test coverage relatively between round, when test coverage raises gradually, judges that project is towards good future development; When test coverage declines, when getting rid of test reason, retreating appears in determining program quality, needs the attention causing project manager.
CN201510694988.0A 2015-10-23 2015-10-23 A kind of method based on Program instrumentation statistical test coverage rate Pending CN105224460A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510694988.0A CN105224460A (en) 2015-10-23 2015-10-23 A kind of method based on Program instrumentation statistical test coverage rate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510694988.0A CN105224460A (en) 2015-10-23 2015-10-23 A kind of method based on Program instrumentation statistical test coverage rate

Publications (1)

Publication Number Publication Date
CN105224460A true CN105224460A (en) 2016-01-06

Family

ID=54993445

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510694988.0A Pending CN105224460A (en) 2015-10-23 2015-10-23 A kind of method based on Program instrumentation statistical test coverage rate

Country Status (1)

Country Link
CN (1) CN105224460A (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106294163A (en) * 2016-08-12 2017-01-04 北京航空航天大学 A kind of code branch covering rate detection method of lightweight
CN109542789A (en) * 2018-11-26 2019-03-29 泰康保险集团股份有限公司 A kind of code coverage statistical method and device
WO2019071929A1 (en) * 2017-10-09 2019-04-18 武汉斗鱼网络科技有限公司 Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device
CN110580214A (en) * 2018-06-07 2019-12-17 百度在线网络技术(北京)有限公司 application program test evaluation method and device, storage medium and terminal equipment
CN112597064A (en) * 2021-03-02 2021-04-02 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN112597007A (en) * 2020-12-14 2021-04-02 中国航发控制系统研究所 Embedded software integration test integrity analysis method
CN113094289A (en) * 2021-05-18 2021-07-09 绿漫科技有限公司 Method for realizing interface coverage rate statistics through python language
US11086759B2 (en) 2018-09-27 2021-08-10 SeaLights Technologies LTD System and method for probe injection for code coverage
US11093374B2 (en) 2016-08-09 2021-08-17 SeaLights Technologies LTD System and method for continuous testing and delivery of software
CN113392033A (en) * 2021-08-17 2021-09-14 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate
CN113535559A (en) * 2021-07-14 2021-10-22 杭州默安科技有限公司 Application interface coverage rate statistical method and system
US11573885B1 (en) 2019-09-26 2023-02-07 SeaLights Technologies LTD System and method for test selection according to test impact analytics

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7334161B2 (en) * 2004-04-30 2008-02-19 Arm Limited Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus
CN102103538A (en) * 2011-02-22 2011-06-22 南京航空航天大学 Method for testing palletizing robot control software based on Agent
CN102419728A (en) * 2011-11-01 2012-04-18 北京邮电大学 Method for determining software test process sufficiency based on coverage rate quantitative indicators

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7334161B2 (en) * 2004-04-30 2008-02-19 Arm Limited Breakpoint logic unit, debug logic and breakpoint method for a data processing apparatus
CN102103538A (en) * 2011-02-22 2011-06-22 南京航空航天大学 Method for testing palletizing robot control software based on Agent
CN102419728A (en) * 2011-11-01 2012-04-18 北京邮电大学 Method for determining software test process sufficiency based on coverage rate quantitative indicators

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11093374B2 (en) 2016-08-09 2021-08-17 SeaLights Technologies LTD System and method for continuous testing and delivery of software
US11775416B2 (en) 2016-08-09 2023-10-03 SeaLights Technologies LTD System and method for continuous testing and delivery of software
CN106294163A (en) * 2016-08-12 2017-01-04 北京航空航天大学 A kind of code branch covering rate detection method of lightweight
WO2019071929A1 (en) * 2017-10-09 2019-04-18 武汉斗鱼网络科技有限公司 Unit testing code coverage (cc) generation method and apparatus, readable storage medium, and device
CN110580214A (en) * 2018-06-07 2019-12-17 百度在线网络技术(北京)有限公司 application program test evaluation method and device, storage medium and terminal equipment
US11847041B2 (en) 2018-09-27 2023-12-19 Sealights Technologies Ltd. System and method for probe injection for code coverage
US11086759B2 (en) 2018-09-27 2021-08-10 SeaLights Technologies LTD System and method for probe injection for code coverage
CN109542789B (en) * 2018-11-26 2022-03-25 泰康保险集团股份有限公司 Code coverage rate statistical method and device
CN109542789A (en) * 2018-11-26 2019-03-29 泰康保险集团股份有限公司 A kind of code coverage statistical method and device
US11573885B1 (en) 2019-09-26 2023-02-07 SeaLights Technologies LTD System and method for test selection according to test impact analytics
CN112597007A (en) * 2020-12-14 2021-04-02 中国航发控制系统研究所 Embedded software integration test integrity analysis method
CN112597007B (en) * 2020-12-14 2023-10-03 中国航发控制系统研究所 Integrated test integrity analysis method for embedded software
CN112597064A (en) * 2021-03-02 2021-04-02 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN112597064B (en) * 2021-03-02 2021-06-29 芯华章科技股份有限公司 Method for simulating program, electronic device and storage medium
CN113094289A (en) * 2021-05-18 2021-07-09 绿漫科技有限公司 Method for realizing interface coverage rate statistics through python language
CN113535559A (en) * 2021-07-14 2021-10-22 杭州默安科技有限公司 Application interface coverage rate statistical method and system
CN113392033B (en) * 2021-08-17 2022-03-08 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate
CN113392033A (en) * 2021-08-17 2021-09-14 北京安普诺信息技术有限公司 Method and device for determining passive IAST test API coverage rate

Similar Documents

Publication Publication Date Title
CN105224460A (en) A kind of method based on Program instrumentation statistical test coverage rate
EP1899876B1 (en) System and method for using model analysis to generate directed test vectors
US5500941A (en) Optimum functional test method to determine the quality of a software system embedded in a large electronic system
CN107301119B (en) Method and device for analyzing IT fault root cause by utilizing time sequence correlation
CN102682229B (en) Malicious code behavior detection method based on virtualization technology
Tahat et al. Regression test suite prioritization using system models
WO1997002528A1 (en) Method and system for an architecture based analysis of software quality
CN102445941B (en) Method for automatically determining and analyzing interoperability test results of on-board equipment of train control system
CN107478455A (en) A kind of Censoring reliability test method suitable for Weibull distribution type product
CN111756582A (en) Service chain monitoring method based on NFV log alarm
CN102999417B (en) Automatic test management system and method
CN101697144A (en) Software test method and software test device
CN101482596B (en) Fast fault recognition method for multifactor industrial system
CN109858097A (en) A kind of spacecraft single machine test assessment methods of sampling
CN114691521A (en) Software testing platform based on artificial intelligence
CN110059010A (en) The buffer overflow detection method with fuzz testing is executed based on dynamic symbol
CN116136950B (en) Chip verification method, device, system, electronic equipment and storage medium
CN105653455A (en) Program vulnerability detection method and detection system
CN112506802A (en) Test data management method and system
CN116954624A (en) Compiling method based on software development kit, software development system and server
CN109359047B (en) Software testing sufficiency measuring method based on system characteristic state
Savor et al. Toward automatic detection of software failures
CN112798944B (en) FPGA hardware error attribution analysis method based on online real-time data
CN113079061B (en) Internet of things performance testing method and system
CN105574312A (en) Method and system for engineering change data analysis

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20160106