CN102855185A - Pair-wise test method based on priority - Google Patents

Abstract

The invention discloses a pair-wise test method based on priority and belongs to the technical field of software testing. The method includes steps that rules are determined through the priority, and a priority value of a dereferencing of each parameter to be tested is defined; a greedy algorithm is utilized to obtain M candidate test cases according to an one dimensional expansion strategy and the priority of the dereferencing of each parameter to be tested; the candidate test cases are coded so that an initial population can be obtained, then the initial population is evolved by a provided genetic algorithm, when the genetic algorithm stops, optimum individual is selected among the initial population, and the optimum individual is added into a test case set; and certain times of executing the steps are limited according to test conditions, and the testing is performed according to the sequence of test cases obtained in the test case set during testing. By means of the method, problems that under the condition of limited resources, the key parameters and combination can not be fully tested, the time of generating the test cases is excessively long, the defect detection rate can not be passed and the like are solved.

Description

A kind of paired combined test method of Priority-based

Technical field

The present invention relates to a kind of paired combined test method of Priority-based, the particularly definition of systematic parameter value priority and generate method for the set of uses case of combined test according to priority belongs to the technical field of software test.

Background technology

Software test is the key link that makes up high trusted software.Statistics shows, this link generally accounts for more than 50% of software development total cost, and effectively method of testing is to reduce the key of software development cost.It is more and more huge and complicated that present computer system is just becoming, and often has more input parameter, and each parameter may have a plurality of different values or equivalence class to divide.The most sufficient method of testing is to design the test use cases that covers all combinations between parameter, but the test use cases scale that produces is often too huge, can't accept on cost.For example: to an examining system for the treatment of with k parameter, these parameters have respectively v ₁, v ₂..., v _kIndividual possibility value, this system of Complete test needs

Individual test case.For general system under test (SUT), this number of combinations is a very huge numeral.How therefrom selecting a less subset of scale is very important problem during test case generates as test use cases.On test performance and cost one compromise is exactly combined test.

Because according to the observation, for a lot of application programs, a lot of program errors all are that the interaction by a few parameters causes.For example: Kuhn and Reilly have analyzed the error reporting record of Mozilla browser, and the interaction that find to surpass 70% mistake and be by certain two parameter triggers, and surpass 90% mistake and be by 3 to interact with interior parameter and cause.Like this, we can select some test cases so that for the individual parameter of any t (t is a little positive integer, generally is 2 or 3), this t parameter might value combination covered by a test case at least.We claim that this test philosophy is the t combined test.Especially, when the t value is 2, this 2 combined tests paired combined test that is otherwise known as.

The below illustrates the combined test method with a simple example.Table 1 has been described an e-commerce system, and there are 4 parameters in this system, and each parameter has 3 optional values, and this system of Complete test needs 34=81 test case.Adopt paired combined test criterion, only need 9 test cases in the table 2 during test, can cover all value combinations of any two parameters.

Four parameter systems of table 1

Table 2 is the combined test use-case in pairs

In pairs the combined test method is very effective in system testing, because for the system that k parameter arranged, finishes the needed minimum test case number of paired combined test and be according to the logarithm level growth of k.The people such as Kuhn of American National Standard and technological associations (NIST) adopt 4 software systems to study the wrong recall rate of combined test.Its result shows, the wrong recall rate of combined test surpasses 80% in pairs.The research of Grindal points out, in pairs the combined test method have model simple, to the tester require low, can effectively process the fairly large characteristics such as testing requirement, be a kind of feasible practical testing scheme.

A big chunk research of classical combined test is to utilize traditional constraint solving or optimization method to directly search the covering array.Since this complex nature of the problem be NP completely, most method all is local search algorithm, these methods can not guarantee to obtain optimum solution, but the processing time is relatively less.These methods mainly comprise greedy algorithm and heuristic search.In addition, there is a small amount of research to adopt the global search algorithm, can obtains optimum solution to the problem of certain scale.

But in actual test job, if the limited time of implementation of test cases, the test use cases that generates with the conventional combination method of testing is difficult to carry out fully under limited resources.At this moment, the tester can wish at first to carry out those critical test cases, under limited resource constraint, come order to carry out them according to the priority of test case in other words, even so that test Halfway Stopping, the test case that importance is high also is performed, improve testing efficiency with this, guarantee the defects detection rate.Otherwise, even the very large cost of flower finds optimum test use cases and has ignored the priority of test case, in the situation that test condition limited be easy to cause key parameter can not test, wasting manpower and material resources, defects detection rate the problem such as do not pass through.Therefore, we wish to treat the value of each parameter of test macro and determine its priority, then obtain one with the test use cases of priority according to priority.The order of concentrating according to test case in test is tested one by one, even fail to have surveyed all parameters or combination, also can farthest guarantee the quality of software systems.

Summary of the invention

Goal of the invention: the deficiency that the present invention is directed to test case shortage precedence relationship in the existing combined test method, a kind of paired combined test method of Priority-based has been proposed, by the definition priority criteria, for each value of system's parameter to be measured is established priority valve, obtain test use cases according to greedy algorithm and genetic algorithm again, thereby realized that test case generates by priority, guaranteed as far as possible tested arriving of key parameter, promoted simultaneously the formation speed of test case.

Technical scheme: a kind of paired combined test method of Priority-based is comprised of priority definition and test case generating algorithm two parts.Wherein: the definition of priority comprises the calculating and preferential built-up pattern of priority valve, and priority valve refers to the priority valve that the formula that proposes by the present invention according to the priority key element calculates certain value of certain parameter; Preferential built-up pattern refers to the ordered cover matrix, i.e. the mathematical model of test use cases of the presently claimed invention.The test case generating algorithm generates orderly test use cases according to the priority valve of each value of parameter to be measured.

Described priority valve calculates influence factor and sees Table 1.The present invention encloses numerical value between one 0 to 1 as its weights to each value of parameter, and these weights are used for representing its precedence information, and its priority of the larger expression of weights is higher.The standard of weight setting is dependent on code coverage, test case executory cost tolerance, the code domain of revising recently, revises frequency, user's usage frequency, span etc.

Table 1 priority valve influence factor

Each factor is occupied certain ratio in priority valve, we calculate the priority valve of a value of certain parameter according to formula 1:

$w=c\×{\mathrm{\η}}_1+\frac{p_{\max }-p}{p_{\max }-p_{\min }}\×{\mathrm{\η}}_2+r\×{\mathrm{\η}}_3+\frac{m*}{m_{\max }}\×{\mathrm{\η}}_4+\frac{u*}{u_{\max }}\×{\mathrm{\η}}_5+\frac{v*}{v_{\max }}\×{\mathrm{\η}}_6$ Formula 1

Wherein, η ₁To η ₆Represent respectively the ratio of each factor in total priority valve, be 0 to 1 decimal, and η ₁+ η ₂+ η ₃+ η ₄+ η ₅+ η ₆=1; W represents the priority valve of certain parameter value; C represents code coverage, is a decimal in 0 to 1; P represents the testing cost of this test case, p _MaxMaximum cost in the expression set of uses case, p _MinThe expression minimum cost; R represents code from being modified to current time measure, the formula 2 below it satisfies; M represents to revise frequency, m ^*Represent this parameter actual modification number of times, m _MaxRevise the modification number of times of the maximum parameter of number of times in the expression system; U represents user's usage frequency, u ^*Represent the discreet value of the actual access times of this parameter, u _MaxThe discreet value of the parameter access times that access times are maximum in the expression system; V represents span, v ^*Represent the actual value number of this parameter, v _MaxThe value number of the parameter that the value number is maximum in the expression system.

$r=e^{-\frac{t}{\mathrm{\α}}}$ Formula 2

Wherein, t revises duration from the last time, and α is constant, and this formula meets the forgetting law curve, and its metric of the code of namely just having revised is 1 to the maximum, along with passage of time diminishes gradually.

After giving weights to each value of each parameter, the weights of any two tuples are that the weights of two values in two tuples are long-pending; The first weights sum of two tuples of appearance that the weights of a test case cover for it.

Described preferential built-up pattern, i.e. ordered cover matrix, it is defined as: satisfy the in pairs ordered cover matrix of combination coverage criteria, be one and cover array, and it satisfies:

(1) order of test case according to priority is worth descending sort;

(2) to any top n of this test set, their weights summation is maximum as much as possible, namely can not find another and covers array, and the weights summation of its top n test case is larger.

Described paired combination coverage criteria is defined as: suppose test use cases TS _{M * k}The matrix of the capable k row of m, if the two tuples set that i is listed as and the j row consist of in the matrix all comprises set V _i* V _j(i parameter, V are shown in the i tabulation to middle all elements in the matrix _iRepresent all value set of i parameter, V _i* V _jRepresent the two tuples set of all value combinations of i and j parameter), then claim TS to meet paired combination coverage criteria.

The Greedy strategy of described greedy algorithm is defined as: choose the value of a parameter, so that the compound weights of all two capped tuples that it forms with fixing parameter are maximum.

The coded system of described genetic algorithm is defined as: adopt the binary coding mode, the possible value number of a parameter f is t, if 2n-1＜t≤2n, the coding figure place of this parameter is n so.Such as following genetic algorithm encoding, be provided with a system and formed by 4 parameters: (f1, f2, f3, f4), the parameter value number is followed successively by 2,2, and 3,4; Can adopt length is 6 binary coding representation: f1 takies a b0; F2 takies a b1; F3 takies a b2 and b3; F4 takies a b4 and b5.For f3, front 3 codings 00,01,10 are respectively v3,0, v3,1, v3,2; Coding 11 can be chosen that of weights maximum in them.If the coded representation number is m, the parameter value number is n, and m＞n, then the parameter value of m-n weights maximum before rear m-n the coded representation.

Genetic algorithm encoding

The fitness function of described genetic algorithm is defined as: with the compound weights of test case as fitness, i.e. the weights sum of its two tuple that occur first of covering, fitness function is used for calculating these compound weights.

The single-point intersection of described genetic algorithm is defined as: two coded sequences are pointed out disconnection same, and the part intersection that their disconnect is spliced.

The paired combined test method of Priority-based comprises the steps:

Step 1 according to the priority computing formula, calculates the priority valve of each value of each parameter, obtains two tuples and weights thereof that the value by all parameters forms, and they are put into not capped two tuples set Uncover;

Step 2 according to greedy algorithm, is tried to achieve M candidate's test case from Uncover is concentrated according to the priority valve situation of parameter value;

Step 3 according to genetic algorithm, with the advancing operation of going forward side by side of step 2 gained candidate test case coding, when genetic algorithm stops, being chosen optimum individual and is added test case and concentrate, and concentrates from Uncover and to leave out two capped tuples;

If Uncover collection not sky and test resource still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4 according to the priority of gained test use cases, is tested one by one to the parameter in the system.

In the paired combined test method of described Priority-based, the implementation step of step 2 is as follows:

Step 2-1 concentrates from Uncover and to choose two large tuple t of M before the weights _i(1≤i≤M if Uncover concentrates not enough M of two tuple numbers, then all chooses);

Step 2-2 is according to two tuple t _iDetermine candidate's test case test _iThe value of two parameters in (the i definition is identical with step 2-1);

Step 2-3, the loose parameter remaining to M test case determined value according to Greedy strategy in order successively, obtains at last test _i, ending step 2.

In the paired combined test method of described Priority-based, the implementation step of step 3 is as follows:

Step 3-1 encodes to M the test case that step 2 obtains;

Step 3-2 utilizes fitness function to obtain the fitness of these test cases;

If the evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _MaxThe individuality that the fitness of ratio is higher is chosen rear P simultaneously _MinThe individuality that the fitness of ratio is lower is participated in follow-on evolutionary process;

Step 3-4, the individuality that step 3-3 is chosen carries out the single-point intersection;

Step 3-5 does the scale-of-two inversion operation to certain position in the sequence to the individuality of step 3-4 gained at random by probability P m; Turn to step 3-2;

Step 3-6, the individuality of choosing the fitness optimum adds test case to be concentrated, and concentrates from Uncover and to leave out two capped tuples, ending step 3.

Beneficial effect: compared with prior art, the paired combined test method of Priority-based provided by the invention adopts the definition of priority so that the gained test case can be in the limited resources situation test macro key parameter effectively; Add genetic algorithm by greedy algorithm, generation that can the accelerated test use-case reduces the test case manufacturing cost.

Description of drawings

Fig. 1 is the process flow diagram of the embodiment of the invention;

Fig. 2 is the process flow diagram of greedy algorithm in the embodiment of the invention;

Fig. 3 is the process flow diagram of genetic algorithm in the embodiment of the invention.

Embodiment

Below in conjunction with specific embodiment, further illustrate the present invention, should understand these embodiment only is used for explanation the present invention and is not used in and limits the scope of the invention, after having read the present invention, those skilled in the art all fall within the application's claims limited range to the modification of the various equivalent form of values of the present invention.

As shown in Figure 1, the paired combined test method of the Priority-based of the present embodiment proposition is divided into four steps:

Step 1 is set up priority model.Being defined in the summary of the invention of priority provides.For a system, at first need each value of each parameter to carry out the assessment of preferential influence factor; Then obtain the priority valve of each parameter value according to the priority valve computing formula in the invention; Then the combination of each parameter value is listed and is calculated the priority valve of these two tuples.The below provides priority model concrete operations is set:

If there are A, B, three parameters of C in a system, wherein, parameter A has 4 values (A1, A2, A3, A4), and B parameter has 2 values (B1, B2), and parameters C has 3 values (C1, C2, C3).The value of parameter and preferential influence factor value such as table 1:

The value of table 1 parameter and preferential influence factor value

Can calculate the priority valve of each parameter value according to formula proposed by the invention, formula is as follows:

$w=c\×{\mathrm{\η}}_1+\frac{p_{\max }-p}{p_{\max }-p_{\min }}\×{\mathrm{\η}}_2+r\×{\mathrm{\η}}_3+\frac{m*}{m_{\max }}\×{\mathrm{\η}}_4+\frac{u*}{u_{\max }}\×{\mathrm{\η}}_5+\frac{v*}{v_{\max }}\×{\mathrm{\η}}_6$

Here we are to η 1 to η 6 difference assignment, η 1=0.2, η 2=0.2, η 3=0.15, η 4=0.15, η 5=0.15, η 6=0.15.Here each ratio can be adjusted by actual conditions.The data substitution formula of upper table can be calculated the priority valve of each parameter value.The priority valve situation such as the table 2 that calculate:

The priority valve of table 2 parameter value

After obtaining the priority valve of each parameter value, the combination of parameter value and its weights are obtained, the weights after the combination are the product of two value priority valves, such as table 3:

The combination of table 3 parameter value and priority valve thereof

Owing in some sampling process, having estimation, so final priority valve only need keep the preferential situation that two-decimal can reflect combination, so in table 3, all priority valves that calculate have been done approximate processing.

Priority model set up complete after, to not capped two tuples set Uncover initialization, all binary combination are put into wherein.

Step 2 according to greedy algorithm, is concentrated from Uncover and to be tried to achieve M candidate's test case according to the priority valve situation of parameter value, its algorithm flow as shown in Figure 2, concrete steps are as follows:

Step 2-1 concentrates from Uncover and to choose two large tuple t of M before the weights _i(1≤i≤M if Uncover concentrates not enough M of two tuple numbers, then all chooses);

The Uncover collection is carried out descending sort, the set such as the table 4 that obtain:

Table 4 according to priority is worth the Uncover set of descending

Here we establish M=9, also can adjust according to actual conditions the value of M.Namely concentrate two tuples of choosing front 8 weights maximums from Uncover, be respectively: A4C1, A4C2, B2C1, A1C1, A4C3, B1C1, B2C2, A1C2, B1C2.

Step 2-2 is according to two tuple t _iDetermine candidate's test case test _iThe value of two parameters in (the i definition is identical with step 2-1), namely the value in these 8 two tuples is fixed, and they are as the part of determining in 8 candidate's test cases;

Step 2-3, the loose parameter remaining to M test case determined value according to Greedy strategy in order successively, obtains at last test _i

Greedy strategy of the present invention is the value of choosing a parameter, so that the compound weights of all two capped tuples that it forms with fixing parameter are maximum.It for example, for A4C1, lacks the value of B parameter, therefore can make up with B1 or B2.If with the B1 combination, then its compound weights are:

Weights=0.20+0.32+0.27=0.79 of weights+B1C1 of weights+A4C1 of A4B1

If with the B2 combination, then its compound weights are:

Weights=0.22+0.32+0.29=0.83 of weights+B2C1 of weights+A4C1 of A4B2

Can find out, if make up with B2, then compound weights are larger, therefore select B2 and its combination, produce candidate's test case A4B2C1.Each two tuple is carried out such operation can obtain whole candidate's test cases.

Step 3 is according to genetic algorithm, with the advancing operation of going forward side by side of step 2 gained candidate test case coding, when genetic algorithm stops, choosing optimum individual and add test case and concentrate, and concentrate from Uncover and to leave out two capped tuples, genetic algorithm flow process such as Fig. 3, concrete steps are as follows;

Step 3-1 encodes to 9 test cases that step 2 obtains;

Coding need to be determined binary figure place according to the parameter value number.Have 4 values, B parameter to have 2 values, parameters C that 3 values are arranged such as parameter A, then with 1 bit representation, parameters C 2 bit representations, corresponding coding and value relation are as shown in table 5 with 2 bit representations, B parameter for parameter A:

Table 5 coding and value relation

Parameters C only has 3 values, and coding can represent 4, thus last coded representation that value of priority valve maximum.

Be encoded to 11100 for the such test case of A4B2C1; For a coding as long as can find corresponding value according to table 5,10101 corresponding A 3B2C2 for example.

Step 3-2 utilizes fitness function to obtain the fitness of these test cases, here we with the compound weights of a test case as fitness;

If individual evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _MaxThe individuality that the fitness of ratio is higher is chosen rear P simultaneously _MinThe individuality that the fitness of ratio is lower is participated in follow-on evolutionary process; Here we can choose P _Max, P _MinBe respectively 60% and 20%.

Step 3-4, the individuality that step 3-3 is chosen carries out the single-point intersection; Carry out when intersecting point of crossing of random selection, and the part-structure of two individualities that this point is front or rear exchanges, and generate two new individualities.For example:

Individual A:001 ↑ 11 → 00101 (new individual A ')

Individual B:100 ↑ 01 → 10011 (new individual B ')

Step 3-5 does the scale-of-two inversion operation to certain position in the sequence to the individuality of step 3-4 gained at random by probability P m.Pm is an experience amount, according to the actual conditions adjustment; Turn to step 3-2;

Step 3-6, the individuality of choosing the fitness optimum adds test case to be concentrated, and concentrates from Uncover and to leave out two capped tuples.

For example our optimum individual that obtains by first round genetic algorithm is A4B2C1, and all two tuples that then need it is comprised are concentrated deletion from Uncover, and two tuples that should delete are A4B2, A4C1, B2C1.

If Uncover collection not sky and test resource still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4, the test use cases that obtains according to the present invention is tested one by one to the parameter in the system.Since test case concentrate more forward test case its to meet weights larger, therefore can guarantee that according to this test case sequential testing the system core parameter is tested, by the defects detection rate.

Claims (4)

1. the paired combined test method of a Priority-based, it is characterized in that: comprise that models of priority is set up and test case generates step, wherein: described models of priority establishment step comprises the definition of parameter value priority, the calculating of priority valve, the priority valve of parameter value two tuples and the priority valve of test case; Described test case generation step comprises by greedy algorithm generation candidate's test case and utilizes genetic algorithm to generate optimum test case two parts;

The priority influence factor comprises: code coverage, cost metric, the time measure apart from last change, modification frequency, user's usage frequency, span;

Priority computing formula such as formula 1:

$w=c\×{\mathrm{\η}}_1+\frac{p_{\max }-p}{p_{\max }-p_{\min }}\×{\mathrm{\η}}_2+r\×{\mathrm{\η}}_3+\frac{m*}{m_{\max }}\×{\mathrm{\η}}_4+\frac{u*}{u_{\max }}\×{\mathrm{\η}}_5+\frac{v*}{v_{\max }}\×{\mathrm{\η}}_6$ Formula 1

Wherein, η ₁To η ₆Represent respectively the ratio of each influence factor in total priority valve, be 0 to 1 decimal, and η ₁+ η ₂+ η ₃+ η ₄+ η ₅+ η ₆=1; W represents the priority valve of certain parameter value; C represents code coverage, is a decimal in 0 to 1; P represents the testing cost of this test case, p _MaxMaximum cost in the expression set of uses case, p _MinThe expression minimum cost; R represents code from being modified to current time measure, the formula 2 below it satisfies; M represents to revise frequency, m ^*Represent this parameter actual modification number of times, m _MaxRevise the modification number of times of the maximum parameter of number of times in the expression system; U represents user's usage frequency, u ^*Represent the discreet value of the actual access times of this parameter, u _MaxThe discreet value of the parameter access times that access times are maximum in the expression system; V represents span, v ^*Represent the actual value number of this parameter, v _MaxThe value number of the parameter that the value number is maximum in the expression system;

$r=e^{-\frac{t}{\mathrm{\α}}}$ Formula 2

Wherein, t revises duration from the last time, and α is constant, and this formula meets the forgetting law curve, and its metric of the code of namely just having revised is 1 to the maximum, along with passage of time diminishes gradually;

The priority valve of described parameter value two tuples is that the weights of two values in two tuples are long-pending;

The first weights sum of two tuples of appearance that the priority valve of described test case covers for it;

The Greedy strategy of described greedy algorithm is defined as: choose the value of a parameter, so that the compound weights of all two capped tuples that it forms with fixing parameter are maximum;

The coded system of described genetic algorithm is defined as: adopt the binary coding mode, the possible value number of a parameter f is t, if 2n-1＜t≤2n, the coding figure place of this parameter is n so; If the coded representation number is m, the parameter value number is n, and m＞n, then the parameter value of m-n weights maximum before rear m-n the coded representation;

The fitness function of described genetic algorithm is defined as: with the compound weights of test case as fitness, i.e. the weights sum of its two tuple that occur first of covering, fitness function is used for calculating these compound weights;

The single-point intersection of described genetic algorithm is defined as: two coded sequences are pointed out disconnection same, and the part intersection that their disconnect is spliced.

2. the paired combined test method of Priority-based as claimed in claim 1 is characterized in that, comprises the steps:

Step 1 according to the priority computing formula, calculates the priority valve of each value of each parameter, obtains two tuples and weights thereof that the value by all parameters forms, and they are put into not capped two tuples set Uncover;

Step 2 according to greedy algorithm, is tried to achieve M candidate's test case from Uncover is concentrated according to the priority valve situation of parameter value;

Step 3 according to genetic algorithm, with the advancing operation of going forward side by side of step 2 gained candidate test case coding, when genetic algorithm stops, being chosen optimum individual and is added test case and concentrate, and concentrates from Uncover and to leave out two capped tuples;

If Uncover collection not sky and test resource still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4 according to the priority of gained test use cases, is tested one by one to the parameter in the system.

3. the paired combined test method of Priority-based as claimed in claim 2 is characterized in that, the implementation step of described step 2 is as follows:

Step 2-1 concentrates from Uncover and to choose two large tuple t of M before the weights _i, wherein 1≤i≤M if Uncover concentrates not enough M of two tuple numbers, then all chooses;

Step 2-2 is according to two tuple t _iDetermine candidate's test case test _iIn the value of two parameters, the i definition is identical with step 2-1;

Step 2-3, the loose parameter remaining to M test case determined value according to Greedy strategy in order successively, obtains at last test _i, ending step 2.

4. the paired combined test method of Priority-based according to claim 2 is characterized in that, the implementation step of described step 3 is as follows:

Step 3-1 encodes to M the test case that step 2 obtains;

Step 3-2 utilizes fitness function to obtain the fitness of described test case;

If the evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _MaxThe individuality that the fitness of ratio is higher is chosen rear P simultaneously _MinThe individuality that the fitness of ratio is lower is participated in follow-on evolutionary process;

Step 3-4, the individuality that step 3-3 is chosen carries out the single-point intersection;

Step 3-5 does the scale-of-two inversion operation to certain position in the sequence to the individuality of step 3-4 gained at random by probability P m; Turn to step 3-2;

Step 3-6, the individuality of choosing the fitness optimum adds test case to be concentrated, and concentrates from Uncover and to leave out two capped tuples, ending step 3.

Images