CN102419728B - Method for determining software test process sufficiency based on coverage rate quantitative indicators - Google Patents
Method for determining software test process sufficiency based on coverage rate quantitative indicators Download PDFInfo
- Publication number
- CN102419728B CN102419728B CN201110338915.XA CN201110338915A CN102419728B CN 102419728 B CN102419728 B CN 102419728B CN 201110338915 A CN201110338915 A CN 201110338915A CN 102419728 B CN102419728 B CN 102419728B
- Authority
- CN
- China
- Prior art keywords
- capped
- test
- function
- file
- coverage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The invention provides a method for determining software test process sufficiency based on coverage rate quantitative indicators. The method comprises the following steps of: A, performing precompilation, lexical analysis and syntax analysis on a tested program to acquire the calling relationship diagram of the abstract syntax tree, the control flow graph and the function of the program, and automatically generating a tested unit according to the test requirement of a user; B, extracting a corresponding element set to be covered in the tested unit from the abstract syntax tree and the control flow graph according to a coverage criterion; C, inserting a corresponding probe function into the program of the tested unit according to the coverage criterion; D, dynamically testing the tested unit into which the probe function is inserted by selecting a system automatic executing mode or a user manual executing mode according to the test requirement of the user; and E, acquiring the covered element set of the tested unit according to the returned information of dynamic test execution, and calculating the coverage rate of the tested unit. By the method, the efficiency and the accuracy of a software evaluation test can be improved.
Description
Technical field
The present invention relates to software testing technology, relate in particular to a kind of method of determining software test procedure adequacy based on coverage rate quantizating index, utilize code coverage to determine that as quantizating index whether the test in software test is abundant.
Background technology
In each stage of software unit testing, people always expect to determine with a unified quantizating index adequacy of test process and method, and instruct when finish test with this.Here, coverage rate is quantitatively evaluating standard common in software unit testing, and common coverage criterion comprises that statement covers, judges that the decision condition of covering, Condition Coverage Testing, judgement/Condition Coverage Testing, modification covers, combines covering and path covering etc.But, existing traditional coverage criterion, and be not suitable for each stage of software test.Described test phase, comprises unit testing, integration testing and system testing.In addition, traditional coverage criterion also needs to be further expanded, to solve the new problem in software testing technology evolution.
Summary of the invention
In view of this, fundamental purpose of the present invention is to provide a kind of method of determining software test procedure adequacy based on coverage rate quantizating index, expands the operational phase of traditional coverage criterion, to be suitable for the multiple dynamic testing method in the different test phases of assessment; And expand the coverage criterion that conventional elements is tested, the new criterions such as file covering, method covering and class covering are proposed; And can be according to given coverage criterion, unit under test is carried out to static analysis, obtain element set to be covered, according to user test demand, carry out dynamic test execution, the capped element of statistics, and calculate coverage rate, thereby according to coverage rate quantizating index, determine the adequacy of software test procedure and method, use the coverage rate of multiple coverage criterion statistics to weigh, improve efficiency and the accuracy of assessment test.
For achieving the above object, technical scheme of the present invention is achieved in that
A method of determining software test procedure adequacy based on coverage rate quantizating index, the method mainly comprises:
A, tested program is carried out to precompile, lexical analysis and grammatical analysis, obtain program abstract syntax tree, control the call graph between flow graph and function, and according to user test demand, automatically generate unit under test;
B, according to coverage criterion, from described abstract syntax tree with control flow graph and extract element set accordingly to be covered in unit under test;
C, according to described coverage criterion, in the program of unit under test, insert corresponding probe functions;
D, according to user test demand, the mode that selective system automatically performs or user manually carries out is carried out dynamic test to inserting the unit under test of probe functions;
E, the return message of carrying out according to dynamic test, obtain the capped element set of test cell, and calculate the coverage rate of unit under test.
Wherein, described steps A comprises:
According to the function calling relationship in tested program, tested program is carried out to dividing elements, obtain unit under test; Described unit under test is according to user test demand, the independently tested object that tested procedure division is become, and it is one or more functions, class, file.
Described step B comprises:
For statement coverage criterion, extract in the function of unitunder test each and can reach statement block and cover element as statement, form statement and cover element set;
For branch's coverage criterion, extract in the function of unitunder test, each Ke Da branch covers element as branch, forms branch and covers element set;
For C/DC coverage criterion, to extract in the function of unitunder test, each covers element and comprises: the predicated expressions that can be performed; In predicated expressions, all values not only can travel through whole unit under test thus for very but also can be false simple condition, formed C/DC and covered element set;
For MC/DC coverage criterion, to extract in the function of unitunder test, each covers element and comprises: the predicated expressions that can be performed; In predicated expressions, all simple condition that can independent effect predicated expressions result, travels through whole unit under test thus, forms MC/DC and covers element set;
For path coverage criterion, extract in the function of unitunder test, all reachable paths, as covering element, form path and cover element set;
For function coverage criterion, extract in the class (directly forming the situation of class for function) or file (for the situation of the direct configuration file of function) of unitunder test, all functions that are performed cover element as function, and component function covers element set; As long as supposing has a paths to be performed in function, just think that this function can be performed;
For class coverage criterion, in the file of extraction unitunder test, all classes that are performed cover element as class, form class and cover element set;
For file coverage criterion, extract all files that are performed in unitunder test and cover element as file, composing document covers element set; For the situation by the direct composing document of class, as long as there is a class to be performed in this document, just think that this document can be performed; For the situation by the direct composing document of function, as long as there is a function to be performed in this document, just think that this document can be performed.
Described step C comprises:
For statement coverage criterion, in each element of statement covering element set, insert probe functions, to record the capped situation of statement block in implementation of test cases process;
For branch's coverage criterion, in branch, cover in each element of element set and insert branch's probe functions, to record the capped situation of branch in implementation of test cases process;
For C/DC coverage criterion, in each element of C/DC covering element set, insert probe functions, in can carrying out judgement, each inserts branch's probe functions, in correspondence is judged, all values not only can be as very but also can insert condition probe functions as false simple condition place, to record the capped situation of C/DC element in implementation of test cases process;
For MC/DC coverage criterion, in covering each element in element set, MC/DC inserts probe functions; At each, can carry out judgement place and insert branch's probe functions, all simple condition places of can independent effect judging in correspondence is judged insert condition probe functions, to record the capped situation of MC/DC element in implementation of test cases process;
For path coverage criterion, based on statement, cover and add up, so need not additionally insert probe functions;
For function coverage criterion, based on path, cover and add up, so need not additionally insert probe functions;
For class coverage criterion, based on function, cover and add up, so need not additionally insert probe functions;
For file, cover accurate side, based on class or function covering, add up, so need not additionally insert probe functions.
Described step D comprises:
For system, automatically perform, system generates the test case of unit under test automatically, generates and drives file, call one by one test case, by driving file and inserting probe functions unit under test place file chaining afterwards, generate executable file, then move executable file, carry out test execution;
For user, manually carry out, by user, manually to pretreated, treat that measuring program generates possible pile function, test case and driving file, and will drive the unit under test place file after file and described insertion probe functions to link, generate executable file operation, carry out test execution.
Described step e specifically comprises:
E1, utilize the return message of the probe functions of tested program, obtain capped element set;
E2, analysis cover result, mode by graphical interfaces identifies and in source code, has covered element and do not covered element, and calculates sentence covering rate, branch covering rate, C/DC coverage rate, MC/DC coverage rate, path coverage, function coverage rate, class coverage rate, the file coverage rate of unit under test.
Further, described step e 1 comprises:
E11, for statement coverage criterion, probe functions is returned to numeral corresponding to all capped statement block that unit under test statement in test process covers set;
E12, for branch's coverage criterion, probe functions is returned to all capped numeral corresponding to branch that unit under test branch in test process covers set;
E13, for C/DC coverage criterion, probe functions is returned to numeral corresponding to the capped element that unit under test C/DC in test process covers set (all capped judgements and judge in all capped simple condition);
E14, for MC/DC coverage criterion, probe functions is returned to numeral corresponding to the capped element that unit under test MC/DC in test process covers set (all capped judgements, and in judging all capped simple condition of can independent effect judging);
E15, for path coverage criterion, according to the statement obtaining in step e 11, cover the capped statement block of set, according to the capped principle of all statement blocks on capped and if only if this path of a paths, extract all capped path of unitunder test;
E16, for function coverage criterion, according to the path obtaining in step e 15, cover capped path in set, according to having the capped principle of a paths at least in capped and if only if this function of a function, extract all capped function (situation about directly being formed by class for file) of tested class or all capped function (situation about directly being formed by function for file) in file;
E17, for class coverage criterion, according to the capped function obtaining in step e 16, according to a class, capped and if only if has a principle that function is capped at least in this type of, extracts all capped class of tested file;
E18, for file coverage criterion, if file directly consists of class, according to the capped class obtaining in step e 17, according to having a principle that class is capped at least in capped and if only if this file of a file, or according to the capped function obtaining in E16, according to having a principle that function is capped at least in capped and if only if this file of a file, extract all capped file of tested engineering.
The method of determining software test procedure adequacy based on coverage rate quantizating index provided by the present invention, has the following advantages:
The method has been expanded the operational phase of traditional coverage rate on the one hand, is applicable to assess the multiple dynamic testing method in different test phases, and test phase comprises unit testing, integration testing and system testing; The coverage criterion of having expanded on the other hand conventional elements test, proposes the criterions such as file covering, method covering and class covering.And can be according to given coverage criterion, unit under test is carried out to static analysis, obtain element set to be covered, also can carry out dynamic test execution according to user test demand, the capped element of statistics, and calculate coverage rate, thereby can determine according to coverage rate quantizating index the adequacy of software test procedure and method, by using the coverage rate of multiple coverage criterion statistics to weigh, improve efficiency and the accuracy of assessment test.
Accompanying drawing explanation
Fig. 1 the present invention is based on the method flow schematic diagram that coverage rate quantizating index is determined software test procedure adequacy;
Fig. 2 is the method flow schematic diagram that software test procedure of the present invention carries out automatic unit division;
Fig. 3 is the method flow schematic diagram that software test procedure of the present invention carries out automatic test execution.
Embodiment
Below in conjunction with accompanying drawing and embodiments of the invention, method of the present invention is described in further detail.
In white box automation software testing, the unified calculation method of multiple coverage rate can adopt different testing tools and method of testing Dynamic Execution tested program, the method of definite software test procedure adequacy provided by the invention, whether fully a kind of coverage rate by automation software testing determines carried out software test procedure means as quantizating index, its basic thought is: read and treat measuring program, it is carried out to static analysis, according to user test demand, tested program is divided into test cell.Wherein, when selective system automatically performs, system, using single function as a unit under test, when selecting to carry out manually, is selected to need one or more functions, class or the file of test by user, and system is using it as a unit under test.Then by user, select coverage criterion, according to the abstract syntax tree of the unit under test generating and control flow graph, extract element set to be covered, the program of unit under test is inserted to corresponding probe functions, can adopt automatically and carry out dynamic test execution with artificial two kinds of modes, according to execution, return results, obtain capped element, and then calculate the coverage rate of unit under test.
Fig. 1 the present invention is based on the method flow schematic diagram that coverage rate quantizating index is determined software test procedure adequacy, as shown in Figure 1, mainly comprises the steps:
Step 1, tested program is carried out to precompile, lexical analysis and grammatical analysis, obtain program abstract syntax tree, control the call graph between flow graph and function, and according to user test demand, automatically generate unit under test.
Described step 1, specifically comprises:
Step 11, according to the function calling relationship in tested program, tested program is carried out to dividing elements, obtain unit under test.
Described unit under test refers to according to user test demand, by the independently tested object of tested procedure division one-tenth, can be one or more functions, class, file etc.Tested engineering be take unit under test as unit divides, and adopts while automatically dividing, and single function is as a unit under test, while adopting manual intervention to divide, by user, specifying forming of each unit under test, can be several functions, class, file, or their combination in any.
Described step 11 further comprises:
Step 111, while automatically dividing, system is using single function as a unit under test.
The schematic flow sheet that automatic unit is divided, as shown in Figure 2:
Take tested program S as example, it comprises 3 function f 1 (type1 v1), f2 (type2 v2), f3 (type3 v3), type1, type2, type3 is parameter type, it is carried out after static analysis, obtain function calling relationship, extract the input and output of each function, each is had to the function of input as a unit under test, hence one can see that, automatically the unit under test of dividing has 3, i.e. { F1, F2, F3}, wherein unit under test F1 is expressed as < f1 (type1 v1), the input/output information > of f1, unit under test F2 is expressed as < f2 (type2 v2), the input/output information > of f2, unit under test F3 is expressed as < f3 (type3 v3), the input/output information > of f3.
When step 112, manual intervention are divided, several functions, class, the file of by user, specifying each unit under test to comprise, or their combination in any, and circulation carries out, until that testing engineering is all divided is complete, obtain the unit under test set of testing engineering.
Step 2, according to coverage criterion (comprising statement, branch, C/DC, MC/DC, path, function, class, file etc.), from abstract syntax tree with control flow graph and extract element set accordingly to be covered in unit under test.It specifically comprises:
Step 21, for statement coverage criterion, extract in the function of unitunder test each and can reach statement block and cover element as statement, form statement and cover element set.If a certain statement wherein of statement that belongs to same is capped, all statements of this piece all can be performed.
Step 22, for branch's coverage criterion, extract in the function of unitunder test, each Ke Da branch (corresponding one by one with the result that can be performed judgement) covers element as branch, forms branch's covering element set.
For example, corresponding on the control flow graph of unit under test, all directed edges are branch and cover element set.
Step 23, for C/DC coverage criterion, extract in the function of unitunder test, each covers element and comprises:
1) predicated expressions that can be performed;
2), in predicated expressions, all values not only can travel through whole unit under test thus for very but also can be false simple condition, formed C/DC and covered element set.
For example: corresponding on the control flow graph of unit under test, be expressed as:
1) directed edge E covers element as a C/DC;
2) when E is the Yi Tiao branch of decision node N, <v> is that a C/DC covers element, wherein v is one group of predicate value of the predicated expressions of N, it be by predicated expressions meet C/DC coverage criterion require determined.
Step 24, for MC/DC coverage criterion, extract in the function of unitunder test, each covers element and comprises:
1) predicated expressions that can be performed;
2), in predicated expressions, all simple condition of can independent effect judging, travels through whole unit under test thus, forms MC/DC and covers element set.
For example: corresponding on the control flow graph of unit under test, be expressed as: get and control a directed edge E on flow graph:
1), when branch that E is non-decision node, E covers element as a MC/DC;
2) when E is the Yi Tiao branch of decision node N, <E, v> covers element as MC/DC, and wherein v is one group of predicate value of the predicated expressions of N, it be by predicated expressions meet MC/DC coverage criterion require determined.
Step 25, for path coverage criterion, extract in the function of unitunder test, all reachable paths, as covering element, form path and cover element set.
For example: corresponding on the control flow graph of unit under test, be expressed as: extract and control an orderly sequence node and an orderly directed edge sequence on flow graph, make this subgraph have unique input node and unique output node, and input node is function entrance, output node is function outlet.
Step 26, for function coverage criterion, extract in the class (directly forming the situation of class for function) or file (for the situation of the direct configuration file of function) of unitunder test, all functions that are performed cover element as function, and component function covers element set.As long as supposing has a paths to be performed in function, just think that this function can be performed.
Step 27, for class coverage criterion, extract all classes that are performed in the file of unitunder test and, as element, form class and cover element set.As long as supposing has a function to be performed in class, just think that such is performed.
Step 28, for file coverage criterion, extract that in unitunder test, all files that are performed are as element, composing document covers element set.For the situation by the direct composing document of class, as long as suppose have a class to be performed in file, just think that this document can be performed.For the situation by the direct composing document of function, as long as suppose have a function to be performed in file, just think that this document can be performed.
Step 3, according to above-mentioned coverage criterion, in the program of unit under test, insert corresponding probe functions.
With exemplary program statistics.c, illustrate how according to different coverage criterions, to insert probe functions below.Program function is: input a line character, counts respectively the wherein number of English alphabet, space, numeral and other character.Source program is as follows:
#include "stdio.h"
void statistics(char c)
{
int letters=0,space=0,digit=0,others=0;
if(c!='\n')
{
if(c>='a'&&c<='z'||c>='A'&&c<='Z')
letters++;
else if(c==' ')
space++;
else if(c>='0'&&c<='9')
digit++;
else
others++;
}
printf("all in all:char=%d space=%d digit=%d others=%d\n",letters,space,digit,others);
}.
This step 3 specifically comprises:
Step 31, for statement coverage criterion, at statement, cover insertion statement probe functions in each element (be and can reach statement block) of element set.
Statement adopts take the statement plug-in mounting that " piece " be unit, and here, so-called piece is exactly the statement collection of the adjacent maximum of execution continuously in logic.At following node place, insert now statement probe functions: before the beginning of function, if statement block, else statement block, case statement piece, default statement block, label statement block, for statement block, while statement piece, do-while piece, catch statement block, finally statement block, skip instruction.
Statement probe functions void _ Block_Cov (int blockNum) is illustrated in plug-in mounting and puts all statements of corresponding statement block and carried out.The abstract syntax tree root node of a statement block of blockNum unique identification wherein, when while statement piece is performed, the identifier that statement block is corresponding is recorded.
Statement probe functions is achieved as follows:
void_Block_Cov(int blockNum)
{
cppTestSendData(blockNum);
}
Source code is: a=d+b;
Code after plug-in mounting is:
a=d+b;
_Block_Cov(Num);
By above-mentioned analysis, can obtain the probe functions of tested program statistics.c under statement coverage criterion and insert result:
#include "stdio.h"
void statistics(char c)
{
int letters=0,space=0,digit=0,others=0;
_Block_Cov(0);
if(c!='\n')
{
_Block_Cov(1);{
if(c>='a'&&c<='z'||c>='A'&&c<='Z')
{
_Block_Cov(2);letters++;
}
else {
_Block_Cov(3);if(c==' ')
{
_Block_Cov(4);space++;
}
else {
_Block_Cov(5);if(c>='0'&&c<='9')
{
_Block_Cov(6);digit++;
}
else{
_Block_Cov(7);others++;
}
}
}
}
}
printf("all in all:char=%d space=%d digit=%d others=%d\n",letters,space,digit,others);
}
Be designated as instrument_statistics.c, wherein the positive and negative bound of returning to value of probe functions can artificially arrange.
Step 32, for branch's coverage criterion, in branch, cover in each element of element set and insert branch's probe functions.
Probe functions bool _ the Bran_Cov of branch (bool exp, Int PositiveBranchId, int negitiveBranchId) obtains motion time and props up situation, and does not change the actuating logic of source code.When operation, according to the numeral receiving, be positive number or negative, differentiate the value of respective branch.Wherein rreturn value PositiveBranchId and negitiveBranchId represent that respectively branch's value is true and false.
Being achieved as follows of branch's probe functions:
bool _Bran_Cov(bool exp, Int PositiveBranchId, int negitiveBranchId)
{
if(exp)
cppTestSendData(PositiveBranchId);
else
cppTestSendData(negitiveBranchId);
Return exp; // the actuating logic of reprogramming not
}
Source code: if (a & & b)
Plug-in mounting post code: if (_ Branch_Cov (a & & b, trueNum, FalseNum))
By above-mentioned analysis, can obtain the probe functions of tested program statistics.c under branch's coverage criterion and insert result:
void statistics(char c)
{
int letters=0,space=0,digit=0,others=0;
_Block_Cov(0);
if(_Bran_Cov(c!='\n',-2147483647,1))
{
if(_Bran_Cov(c>='a'&&c<='z'||c>='A'&&c<='Z',-2147483646,2))
letters++;
else if(_Bran_Cov(c==' ',-2147483645,3))
space++;
else if(_Bran_Cov(c>='0'&&c<='9',-2147483644,4))
digit++;
else
others++;
}
printf("all in all:char=%d space=%d digit=%d others=%d\n",letters,
space,digit,others);
}
Be designated as instrument_statistics.c, wherein the positive and negative bound of returning to value of probe functions can artificially arrange.
Step 33, for C/DC coverage criterion, in each element of C/DC covering element set, insert probe functions, in each can be performed judgement, insert branch's probe functions, in correspondence is judged, all values not only can be as very but also can be that false simple condition place inserts condition probe functions.
Condition probe functions bool _ Cond_Cov (bool exp, int positiveCondId, int negitiveCondId) obtains the value condition that the operation phase is performed simple condition, and does not change the actuating logic of source code.When operation, according to the numeral receiving, be positive number or negative, differentiate the value of corresponding judgement.Wherein rreturn value positiveCondId and negitiveCondId represent that respectively condition value is true and false.
Being achieved as follows of condition probe:
bool _Cond_Cov(bool exp, Int positiveCondId, int negitiveCondId)
{
if(exp)
cppTestSendData(positiveCondId);
else
cppTestSendData(negitiveCondId);
Return exp; // the actuating logic of reprogramming not
}
In predicated expressions, insert branch's probe also for each conditional expression in predicated expressions inserts condition probe.When dynamic operation, according to the plug-in mounting information receiving, the value that obtains condition in the true and false and branch of branch's value is true and false.
Source code: if (a & & b),
Insert the code after branch's probe and condition probe:
if(_Branch_COV(if(_Cond_Cov(a，trueNum，falseNum)&&_Cond_Cov(b，trueNum，falseNum)),trueNum,FalseNum))
By above-mentioned analysis, can obtain the probe functions of tested program statistics.c under C/DC coverage criterion and insert result:
void statistics(char c)
{
int letters=0,space=0,digit=0,others=0;
_Block_Cov(0);
if(_Bran_Cov(_Cond_Cov(c!='\n',-2147483647,1),-2147483646,2))
{
if(_Bran_Cov(_Cond_Cov(c>='a',-2147483645,3)&&_Cond_Cov(c<='z',-2147483644,4)||_Cond_Cov(c>='A',-2147483643,5)&&_Cond_Cov(c<='Z',-2147483642,6),-2147483641,7))
letters++;
else if(_Bran_Cov(_Cond_Cov(c==' ',-2147483640,8),-2147483639,9))
space++;
else if(_Bran_Cov(_Cond_Cov(c>='0',-2147483638,10)&&_Cond_Cov(c<='9',-2147483637,11),-2147483636,12))
digit++;
else
others++;
}
printf("all in all:char=%d space=%d digit=%d others=%d\n",letters,space,digit,others);
}
Be designated as instrument_statistics.c, wherein the positive and negative bound of returning to value of probe functions can artificially arrange.
Step 34, for MC/DC coverage criterion, in MC/DC covers each element in element set, insert probe functions.At each, can be performed judgement place and insert branch's probe functions, all simple condition places of can independent effect judging in correspondence is judged insert condition probe functions, and method is with step 32 and step 33.
Step 35, for path coverage criterion, based on statement, cover and to add up, so need not additionally insert probe functions.
Step 36, for function coverage criterion, based on path, cover and to add up, so need not additionally insert probe functions.
Step 37, for class coverage criterion, based on function, cover and to add up, so need not additionally insert probe functions.
Step 38, for file, cover accurate side, based on class or function, cover and add up, so need not additionally insert probe functions.
Step 4, according to user test demand, select employing system to automatically perform mode or the manual executive mode of user is carried out dynamic test to inserting the unit under test of probe functions.
Wherein, described step 4 specifically comprises:
Step 41, for system, automatically perform, system generates the test case of unit under test automatically, and is generated executable file, carries out test execution.Automatically the process of test execution as shown in Figure 3, for a test cell, generates corresponding test case library, is used for preserving all effective test cases.This step 41 further comprises:
Step 411, system generate corresponding test case according to the input parameter of coverage criterion and unit under test;
Step 412, generation drive file, for calling one by one test case;
Step 413, will drive the unit under test place file chaining inserting in file and step 3 after probe functions to generate executable file;
Step 414, operation executable file, test result will be recorded and analyze simultaneously, and finally generates an interpretation of result report, by all effective test case generating test use cases storehouse.
For example, for tested program statistics.c, system generates a test case (' b ') according to its input parameter type and scope, and generation drives file driverFile_statistics.c, driverFile_statistics.c and the unit under test file instrument_statistics.c after insertion probe functions are linked, generate executable file test_statistics.exe, then move executable file, and obtain test case library testcase_statistics.mdb.
Step step 42, for user, manually carry out, by user, manually to pretreated, treat that measuring program generates possible pile function, test case and driving file, and link inserting probe functions unit under test place file afterwards in driving file and step 3, generate executable file operation, carry out test execution.
Take source program statistics.c as example, after pre-service, user is according to function information, generate pile function _ STUB_statistics, and then generate stake code, then for example, according to the test case of appointment (' b '), generate driving function _ DRIVER_statistics, call test case, finally the tested program after stake code, driving function, insertion probe functions is linked, generate executable file test_statistics.exe, operation executable file, carries out test execution.
Step 5, the return message of carrying out according to dynamic test, obtain the capped element set of test cell, and calculate the coverage rate of unit under test.It specifically comprises:
Step 51, utilize the return message of the probe functions of tested program, obtain capped element set.
Wherein step 51 further comprises:
Step 511, for statement coverage criterion, probe functions is returned to numeral corresponding to all capped statement block that unit under test statement in test process covers set.
For example: the test case (' b ') of carrying out in step 4, the probe functions of unit under test statistics.c return results into: 0,1,2, hence one can see that probe functions _ Block_Cov (0), _ Block_Cov (1), the statement block that _ Block_Cov (2) identifies is capped.
Step 512, for branch's coverage criterion, probe functions is returned to all capped numeral corresponding to branch that unit under test branch in test process covers set.
The test case (' b ') of for example carrying out in step 4, the probe functions of unit under test statistics.c return results into: 0,-2147483647,-2147483646, hence one can see that probe functions _ Bran_Cov (0), _ Bran_Cov (2147483647), the branch that _ Bran_Cov (2147483646) identifies is capped.
Step 513, for C/DC coverage criterion, probe functions is returned to numeral corresponding to the capped element that unit under test C/DC in test process covers set (all capped judgements, and in judging capped all simple conditions).
Step 514, for MC/DC coverage criterion, probe functions is returned to numeral corresponding to the capped element that unit under test MC/DC in test process covers set (all capped judgements, and in judging all capped simple condition of can independent effect judging).
Step 515, for path coverage criterion, according to the statement obtaining in step e 11, cover the capped statement block of set, according to the capped principle of all statement blocks on capped and if only if this path of a paths, extract all capped path of unitunder test.
Step 516, for function coverage criterion, according to the path obtaining in step e 15, cover capped path in set, according to having the capped principle of a paths at least in capped and if only if this function of a function, extract all capped function (situation about directly being formed by class for file) of tested class or all capped function (situation about directly being formed by function for file) in file.
Step 517, for class coverage criterion, according to the capped function obtaining in step e 16, according to a class, capped and if only if has a principle that function is capped at least in this type of, extracts all capped class of tested file.
Step 518, for file coverage criterion, according to the capped class (situation about directly being formed by class for file) obtaining in step e 17, according to having a principle that class is capped at least in capped and if only if this file of a file, or according to the capped function obtaining in E16 (situation about directly being formed by function for file), according to having a principle that function is capped at least in capped and if only if this file of a file, extract all capped file of tested engineering.
Step 52, analysis cover result, mode by graphical interfaces identifies and in source code, has covered element and do not covered element, and calculates sentence covering rate, branch covering rate, C/DC coverage rate, MC/DC coverage rate, path coverage, function coverage rate, class coverage rate, the file coverage rate of unit under test.
Wherein step 52 further comprises:
The coverage rate that step 521, statistics statement cover, the statement first obtaining in statistic procedure 511 covers the capped statement block element number of element set.Mode by graphical interfaces identifies statement block element capped in source code and not capped statement block element.Then the statement extracting in statistic procedure 21 covers the total number of element of element set, covers the total number of element of element set with capped statement block element number divided by statement, can calculate sentence covering rate.
In above-mentioned formula, variable implication is as follows:
: sentence covering rate
: statement covers capped element number in element set
: statement covers the total number of element of element set
The coverage rate that step 522, statistics branch cover, the branch first obtaining in statistic procedure 512 covers the capped branch's element number of element set.Mode by graphical interfaces identifies branch's element capped in source code and not capped branch's element.Then the branch extracting in statistic procedure 22 covers the total number of element of element set, and the total element number that covers element set divided by branch with capped branch's element number, can calculate branch covering rate.
In above-mentioned formula, variable implication is as follows:
: branch covering rate
: branch covers capped element number in element set
: branch covers the total number of element of element set
The coverage rate that step 523, statistics C/DC cover, the capped simple condition first obtaining in statistic procedure 513 and branch's number.Mode by graphical interfaces identifies branch capped in source code, simple condition and number of times thereof, and not capped Ke Da branch and corresponding simple condition thereof.Then add up the C/DC extracting in described step 23 and cover the total number of element (in C/DC covering set, branch's number+value can be and can be very again false simple condition number * 2) of gathering, the number of times sum that is not repeated value by capped branch's number and simple condition covers the total number of element of gathering divided by C/DC, can calculate C/DC coverage rate.
Above-mentioned formula variable implication is as follows:
: C/DC coverage rate
: C/DC covers capped branch's number in element set
: in C/DC covering element set, simple condition is not repeated the number of times of value
: C/DC covers the total number of branch in element set
: the value in C/DC covering element set can be and can be very again the total number of false simple condition
The coverage rate that step 524, statistics MC/DC cover, the capped simple condition first obtaining in statistic procedure 514 and branch's number.Mode by graphical interfaces identifies branch capped in source code, simple condition and number of times thereof, and not capped Ke Da branch and corresponding simple condition thereof.Then the MC/DC extracting in statistic procedure 24 covers the total number of element (in MC/DC covering set, branch's number+value can be and can be very again false simple condition number * 2) of set, by simple condition, be not repeated the number of times of value and the total number of element that capped branch's number sum covers set divided by MC/DC, can calculate MC/DC coverage rate.
Above-mentioned formula variable implication is as follows:
: MC/DC coverage rate
: MC/DC covers capped branch's number in element set
: in MC/DC covering element set, simple condition is not repeated the number of times of value
: MC/DC covers the total number of branch in element set
: the value in MC/DC covering element set can be and can be very again the total number of false simple condition
The coverage rate that step 525, statistical path cover, first utilizes the capped statement obtaining in step 515 to count capped reachable path number.Mode by graphical interfaces identifies capped path and not capped path in source code.Then the reachable path sum calculating divided by the path elements set from step 25 by capped number of path, can calculate path coverage.
Above-mentioned formula variable implication is as follows:
: path coverage
: path covers capped path number in set
: path covers the total number of element in set
Step 526, the coverage rate covering for function, the function first obtaining in statistic procedure 516 covers the capped function number of set.Mode by graphical interfaces identifies capped function and not capped function in source code.Then the function extracting in statistic procedure 26 covers the total number of set element, by capped function number, divided by function, covers the total number of element in set, can calculate function coverage rate.
Above-mentioned formula variable implication is as follows:
: function coverage rate
: function covers capped element number in set
: function covers the total number of element of set
Step 527, the coverage rate covering for class, the class first obtaining in statistic procedure 517 covers the capped class number of set.Mode by graphical interfaces identifies capped class and not capped class in source code.Then the class extracting in statistic procedure 27 covers the total number of element of element set, by capped class number, divided by the total number that covers class in set, can calculate class coverage rate.
Above-mentioned formula variable implication is as follows:
: class coverage rate
: class covers capped class number in set
: class covers the total number of element in set
Step 528, the coverage rate covering for file, the be performed total number of files being capped in the tested engineering first obtaining in statistic procedure 518.Mode by graphical interfaces identifies be performed file capped in source code and not capped be performed file.Then the total number of be performed file extracting in statistic procedure 28, divided by the total number of the file that can be performed in engineering, can calculate file coverage rate by file number capped in engineering.
Above-mentioned formula variable implication is as follows:
: file coverage rate
: capped file number in tested engineering
: file covers the total number of element in set.
The above, be only preferred embodiment of the present invention, is not intended to limit protection scope of the present invention.
Claims (6)
1. based on coverage rate quantizating index, determine a method for software test procedure adequacy, it is characterized in that, the method mainly comprises:
A, tested program is carried out to precompile, lexical analysis and grammatical analysis, obtain program abstract syntax tree, control the call graph between flow graph and function, and according to user test demand, automatically generate unit under test; Described unit under test is according to user test demand, the independently tested object that tested procedure division is become, and it is one or more functions, class, file;
B, according to coverage criterion, from described abstract syntax tree with control flow graph and extract element set accordingly to be covered in unit under test;
C, according to described coverage criterion, in the program of unit under test, insert corresponding probe functions;
D, according to user test demand, the mode that selective system automatically performs or user manually carries out is carried out dynamic test to inserting the unit under test of probe functions;
E, the return message of carrying out according to dynamic test, obtain the capped element set of test cell, and calculate the coverage rate of unit under test; Comprise: E1, utilize the return message of the probe functions of tested program, obtain capped element set; E2, analysis cover result, mode by graphical interfaces identifies and in source code, has covered element and do not covered element, and calculates sentence covering rate, branch covering rate, C/DC coverage rate, MC/DC coverage rate, path coverage, function coverage rate, class coverage rate, the file coverage rate of unit under test.
2. the method for determining software test procedure adequacy based on coverage rate quantizating index according to claim 1, is characterized in that, described steps A comprises:
According to the function calling relationship in tested program, tested program is carried out to dividing elements, obtain unit under test.
3. the method for determining software test procedure adequacy based on coverage rate quantizating index according to claim 1, is characterized in that, described step B comprises:
For statement coverage criterion, extract in the function of unitunder test each and can reach statement block and cover element as statement, form statement and cover element set;
For branch's coverage criterion, extract in the function of unitunder test, each Ke Da branch covers element as branch, forms branch and covers element set;
For C/DC coverage criterion, to extract in the function of unitunder test, each covers element and comprises: the predicated expressions that can be performed; In predicated expressions, all values not only can travel through whole unit under test thus for very but also can be false simple condition, formed C/DC and covered element set;
For MC/DC coverage criterion, to extract in the function of unitunder test, each covers element and comprises: the predicated expressions that can be performed; In predicated expressions, all simple condition that can independent effect predicated expressions result, travels through whole unit under test thus, forms MC/DC and covers element set;
For path coverage criterion, extract in the function of unitunder test, all reachable paths, as covering element, form path and cover element set;
For function coverage criterion, extract in the class (directly forming the situation of class for function) or file (for the situation of the direct configuration file of function) of unitunder test, all functions that are performed cover element as function, and component function covers element set; As long as supposing has a paths to be performed in function, just think that this function can be performed;
For class coverage criterion, in the file of extraction unitunder test, all classes that are performed cover element as class, form class and cover element set;
For file coverage criterion, extract all files that are performed in unitunder test and cover element as file, composing document covers element set; For the situation by the direct composing document of class, as long as there is a class to be performed in this document, just think that this document can be performed; For the situation by the direct composing document of function, as long as there is a function to be performed in this document, just think that this document can be performed.
4. the method for determining software test procedure adequacy based on coverage rate quantizating index according to claim 1, is characterized in that, described step C comprises:
For statement coverage criterion, in each element of statement covering element set, insert probe functions, to record the capped situation of statement block in implementation of test cases process;
For branch's coverage criterion, in branch, cover in each element of element set and insert branch's probe functions, to record the capped situation of branch in implementation of test cases process;
For C/DC coverage criterion, in each element of C/DC covering element set, insert probe functions, in can carrying out judgement, each inserts branch's probe functions, in correspondence is judged, all values not only can be as very but also can insert condition probe functions as false simple condition place, to record the capped situation of C/DC element in implementation of test cases process;
For MC/DC coverage criterion, in covering each element in element set, MC/DC inserts probe functions; At each, can carry out judgement place and insert branch's probe functions, all simple condition places of can independent effect judging in correspondence is judged insert condition probe functions, to record the capped situation of MC/DC element in implementation of test cases process;
For path coverage criterion, based on statement, cover and add up, so need not additionally insert probe functions;
For function coverage criterion, based on path, cover and add up, so need not additionally insert probe functions;
For class coverage criterion, based on function, cover and add up, so need not additionally insert probe functions;
For file, cover accurate side, based on class or function covering, add up, so need not additionally insert probe functions.
5. according to determining and it is characterized in that the method for software test procedure adequacy based on coverage rate quantizating index described in claim 1 or 4, described step D comprises:
For system, automatically perform, system generates the test case of unit under test automatically, generates and drives file, call one by one test case, by driving file and inserting probe functions unit under test place file chaining afterwards, generate executable file, then move executable file, carry out test execution;
For user, manually carry out, by user, manually to pretreated, treat that measuring program generates possible pile function, test case and driving file, and will drive the unit under test place file after file and described insertion probe functions to link, generate executable file operation, carry out test execution.
6. the method for determining software test procedure adequacy based on coverage rate quantizating index according to claim 1, is characterized in that, described step e 1 comprises:
E11, for statement coverage criterion, probe functions is returned to numeral corresponding to all capped statement block that unit under test statement in test process covers set;
E12, for branch's coverage criterion, probe functions is returned to all capped numeral corresponding to branch that unit under test branch in test process covers set;
E13, for C/DC coverage criterion, probe functions return unit under test C/DC in test process cover the i.e. all capped judgements of capped element of set and judge in numeral corresponding to all capped simple conditions;
E14, for MC/DC coverage criterion, probe functions return unit under test MC/DC in test process cover the i.e. all capped judgements of capped element of set and judge in all capped numerals corresponding to simple condition of can independent effect judging;
E15, for path coverage criterion, according to the statement obtaining in step e 11, cover the capped statement block of set, according to the capped principle of all statement blocks on capped and if only if this path of a paths, extract all capped path of unitunder test;
E16, for function coverage criterion, according to the path obtaining in step e 15, cover capped path in set, according to having the capped principle of a paths at least in capped and if only if this function of a function, extract all capped function of tested class or all capped function in file;
E17, for class coverage criterion, according to the capped function obtaining in step e 16, according to a class, capped and if only if has a principle that function is capped at least in this type of, extracts all capped class of tested file;
E18, for file coverage criterion, if file directly consists of class, according to the capped class obtaining in step e 17, according to having a principle that class is capped at least in capped and if only if this file of a file, or according to the capped function obtaining in E16, according to having a principle that function is capped at least in capped and if only if this file of a file, extract all capped file of tested engineering.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110338915.XA CN102419728B (en) | 2011-11-01 | 2011-11-01 | Method for determining software test process sufficiency based on coverage rate quantitative indicators |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110338915.XA CN102419728B (en) | 2011-11-01 | 2011-11-01 | Method for determining software test process sufficiency based on coverage rate quantitative indicators |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102419728A CN102419728A (en) | 2012-04-18 |
| CN102419728B true CN102419728B (en) | 2014-10-29 |
Family
ID=45944147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110338915.XA Expired - Fee Related CN102419728B (en) | 2011-11-01 | 2011-11-01 | Method for determining software test process sufficiency based on coverage rate quantitative indicators |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102419728B (en) |
Families Citing this family (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103425573B (en) * | 2012-05-24 | 2017-02-08 | 腾讯科技（深圳）有限公司 | Code testing information collecting method and system |
| CN103631707B (en) * | 2012-08-27 | 2018-01-30 | 腾讯科技（深圳）有限公司 | A kind of method and device for obtaining code coverage |
| CN103019933B (en) * | 2012-11-27 | 2016-01-13 | 网宿科技股份有限公司 | Test case coverage automatic identifying method and device |
| CN103593291B (en) * | 2013-11-18 | 2016-03-09 | 北京邮电大学 | For comprising unit test method and the device of multiple function test module |
| CN104077232B (en) * | 2014-07-21 | 2017-02-22 | 上海零一拼装信息技术有限公司 | Testing device and method based on use case and source code two-way tracing |
| CN104317710A (en) * | 2014-10-21 | 2015-01-28 | 北京邮电大学 | Under-test program modification method and system for use in structural testing |
| CN104317723B (en) * | 2014-11-14 | 2017-02-22 | 清华大学 | Method and system for tracking running information of drive program |
| CN104391793B (en) * | 2014-11-27 | 2017-07-14 | 中国联合网络通信集团有限公司 | Testing procedure and test script generation method and device |
| CN104375943A (en) * | 2014-12-11 | 2015-02-25 | 吴翔虎 | Embedded software black-box test case generation method based on static models |
| CN106155892B (en) * | 2015-04-03 | 2020-12-29 | 腾讯科技（深圳）有限公司 | Method for judging application program test coverage and program test equipment |
| CN106155897B (en) * | 2015-04-16 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Service processing method and device |
| CN105224460A (en) * | 2015-10-23 | 2016-01-06 | 浪潮电子信息产业股份有限公司 | A kind of method based on Program instrumentation statistical test coverage rate |
| EP3497574A4 (en) | 2016-08-09 | 2020-05-13 | Sealights Technologies Ltd. | System and method for continuous testing and delivery of software |
| CN107870853A (en) * | 2016-09-27 | 2018-04-03 | 北京京东尚科信息技术有限公司 | The method and device of test program code path coverage |
| CN106569944B (en) * | 2016-10-14 | 2019-12-13 | 上海微小卫星工程中心 | Constraint tree-based satellite-borne software test data analysis method |
| CN106874187B (en) * | 2016-12-29 | 2020-09-11 | 腾讯科技（深圳）有限公司 | Code coverage rate collection method and device |
| CN108804304B (en) * | 2017-05-05 | 2022-06-07 | 北京京东尚科信息技术有限公司 | Test case generation method and device, storage medium and electronic equipment |
| CN107229565B (en) * | 2017-05-31 | 2020-05-01 | 北京京东尚科信息技术有限公司 | Test method and device |
| CN107678967B (en) * | 2017-10-09 | 2020-02-07 | 武汉斗鱼网络科技有限公司 | Unit test coverage rate generation method and device, readable storage medium and equipment |
| CN109992498B (en) * | 2017-12-29 | 2022-12-02 | 北京京东尚科信息技术有限公司 | Test case generation method and system and computer system |
| CN108197036B (en) * | 2018-02-06 | 2021-05-25 | 百度在线网络技术（北京）有限公司 | Method and apparatus for determining coverage information for incremental codes |
| CN108710575B (en) * | 2018-05-23 | 2020-11-24 | 华南理工大学 | Unit test method based on automatic generation of path coverage test case |
| CN110556102B (en) * | 2018-05-30 | 2023-09-05 | 蔚来(安徽)控股有限公司 | Method, apparatus, in-vehicle voice dialogue system, and computer storage medium for intention recognition and execution |
| CN109308260B (en) * | 2018-08-30 | 2021-11-05 | 福建天泉教育科技有限公司 | Method and terminal for automatically generating unit test code |
| CN109376075B (en) * | 2018-09-19 | 2022-04-22 | 奇安信科技集团股份有限公司 | Processing method and device for generating optimal test coverage path of test object |
| US11086759B2 (en) | 2018-09-27 | 2021-08-10 | SeaLights Technologies LTD | System and method for probe injection for code coverage |
| CN111949505A (en) * | 2019-05-14 | 2020-11-17 | 阿里巴巴集团控股有限公司 | Test method, device and equipment |
| CN110348226B (en) * | 2019-07-12 | 2021-06-18 | 北京字节跳动网络技术有限公司 | Engineering file scanning method and device, electronic equipment and storage medium |
| CN110543421B (en) * | 2019-08-31 | 2022-03-29 | 华南理工大学 | Unit test automatic execution method based on test case automatic generation algorithm |
| US11573885B1 (en) | 2019-09-26 | 2023-02-07 | SeaLights Technologies LTD | System and method for test selection according to test impact analytics |
| CN110750459B (en) * | 2019-10-23 | 2023-10-20 | 天津汇缘诚信科技有限公司 | Automatic test case generation and test process management method based on white box analysis |
| CN112905443A (en) * | 2019-12-04 | 2021-06-04 | 阿里巴巴集团控股有限公司 | Test case generation method, device and storage medium |
| CN111858322B (en) * | 2020-07-10 | 2022-01-11 | 中国科学技术大学 | Python language feature automatic identification system and method |
| CN112463607B (en) * | 2020-11-26 | 2024-03-15 | 苏州洞察云信息技术有限公司 | Determination statement value confirmation method, device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833500A (en) * | 2010-04-07 | 2010-09-15 | 南京航空航天大学 | Embedded software intelligent testing method based on Agent |
| CN101916225A (en) * | 2010-09-02 | 2010-12-15 | 于秀山 | Graphical user interface software function coverage testing method |
-
2011
- 2011-11-01 CN CN201110338915.XA patent/CN102419728B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101833500A (en) * | 2010-04-07 | 2010-09-15 | 南京航空航天大学 | Embedded software intelligent testing method based on Agent |
| CN101916225A (en) * | 2010-09-02 | 2010-12-15 | 于秀山 | Graphical user interface software function coverage testing method |
Non-Patent Citations (2)
| Title |
|---|
| 基于软件缺陷模型的测试系统;杨朝红等;《北京邮电大学学报》;20081031;第31卷(第5期);1－3 * |
| 杨朝红等.基于软件缺陷模型的测试系统.《北京邮电大学学报》.2008,第31卷(第5期),1－3. |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102419728A (en) | 2012-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102419728B (en) | Method for determining software test process sufficiency based on coverage rate quantitative indicators | |
| Hartmann et al. | Techniques for selective revalidation | |
| CN101739339B (en) | Program dynamic dependency relation-based software fault positioning method | |
| CN105022691B (en) | A kind of increasingly automated method for testing software based on uml diagram | |
| CN103294594B (en) | A kind of wrong report of the static analysis based on test removing method | |
| CN102521123B (en) | A kind of embedded software test pile pitching method of logic-based execution block | |
| CN110134588B (en) | Test case priority ordering method and test system based on code and combination coverage | |
| CN103019933A (en) | Automatic test case coverage identification method and device | |
| US20070044075A1 (en) | Method for analysis of source code and display of corresponding output through a marking scheme | |
| CN105975392A (en) | Duplicated code detection method and device based on abstract syntax tree | |
| CN110543421A (en) | Unit test automatic execution method based on test case automatic generation algorithm | |
| CN104375941A (en) | Automated evaluation method for binary code coverage of testing case set for executable program | |
| CN101183332A (en) | Method and device for automatically generating testing datasets by program content | |
| Zhou et al. | Extracting configuration knowledge from build files with symbolic analysis | |
| CN103605606A (en) | Embedded software test case batch execution method capable of being automatically convertible | |
| CN106294136B (en) | The online test method and system of performance change between the concurrent program runtime | |
| CN108897678B (en) | Static code detection method, static code detection system and storage device | |
| CN102521135B (en) | The method of testing of linear system and device | |
| US10402309B2 (en) | Code coverage tracking for a microcontroller program | |
| US10324829B2 (en) | Application testing | |
| Liu et al. | Automated pattern-directed refactoring for complex conditional statements | |
| CN110347610B (en) | Embedded code debugging system and method | |
| CN110990282B (en) | Automatic unit testing method | |
| CN105045698A (en) | Method for analyzing chip verification progress by using MATPLOTLIB of PYTHON | |
| CN107741905B (en) | Test case, analytic model thereof, execution method, storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20141029 Termination date: 20201101 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |