CN102855185B - Pair-wise test method based on priority - Google Patents

Abstract

The invention discloses a pair-wise test method based on priority and belongs to the technical field of software testing. The method includes steps that rules are determined through the priority, and a priority value of a dereferencing of each parameter to be tested is defined; a greedy algorithm is utilized to obtain M candidate test cases according to an one dimensional expansion strategy and the priority of the dereferencing of each parameter to be tested; the candidate test cases are coded so that an initial population can be obtained, then the initial population is evolved by a provided genetic algorithm, when the genetic algorithm stops, optimum individual is selected among the initial population, and the optimum individual is added into a test case set; and certain times of executing the steps are limited according to test conditions, and the testing is performed according to the sequence of test cases obtained in the test case set during testing. By means of the method, problems that under the condition of limited resources, the key parameters and combination can not be fully tested, the time of generating the test cases is excessively long, the defect detection rate can not be passed and the like are solved.

Description

A kind of Pair-wise test method based on priority

Technical field

The present invention relates to a kind of Pair-wise test method based on priority, particularly the method for the definition of systematic parameter value priority and the set of uses case according to priority generation confession combined test, belongs to the technical field of software test.

Background technology

Software test is the key link building high-confidence software.Statistics shows, this link generally accounts for more than 50% of software development total cost, and effective method of testing reduces the key of software development cost.Present computer system is just becoming more and more huge and complicated, often has more input parameter, and each parameter may have multiple different value or equivalence class partition.The most sufficient method of testing designs the test use cases covering all combinations between parameter, but the test use cases scale produced is often too huge, and cost cannot accept.Such as: to one there is k parameter treat examining system, these parameters have v respectively ₁, v ₂..., v _kindividual possibility value, this system of Complete test needs individual test case.For general system under test (SUT), this number of combinations is a very huge numeral.The subset how therefrom selecting scale less is a very important problem in Test cases technology as test use cases.One on test performance and cost compromise is exactly combined test.

Because according to the observation, for a lot of application program, a lot of program error is all caused by the interaction of a few parameters.Such as: Kuhn and Reilly analyzes the error reporting record of Mozilla browser, find that mistake more than 70% is triggered by the interaction of certain two parameter, the mistake more than 90% to be interacted by the parameter within 3 and initiation.Like this, we can select some test cases, make for any t (t is a little positive integer, is generally 2 or 3) individual parameter, this t parameter the combination of likely value at least covered by a test case.We claim this test philosophy to be t combined test.Especially, when t value is 2, this 2 combined tests be otherwise known as pair-wise combination test.

With a simple example, combined test method is described below.Table 1 describes an e-commerce system, and this system has 4 parameters, and each parameter has 3 selectable value, and this system of Complete test needs 34=81 test case.Adopt pair-wise combination test philosophy, only need 9 test cases in table 2 during test, all valued combinations of any two parameters can be covered.

Table 1 four parameter systems

Table 2 pair-wise combination test case

Pair-wise test method is very effective in system testing, because for the system having k parameter, the minimum test case number completed required for pair-wise combination test increases according to the Logarithmic degree of k.The people such as the Kuhn of American National Standard and technological associations (NIST) adopt 4 software systems to have studied the wrong recall rate of combined test.Its result shows, the wrong recall rate of pair-wise combination test is more than 80%.The research of Grindal is pointed out, Pair-wise test method have model simple, to tester require low, can effectively process the features such as fairly large testing requirement, be a kind of feasible practical testing scheme.

A big chunk research of classical combined test utilizes traditional constraint solving or optimization method to directly search covering array.Due to this complex nature of the problem be NP completely, most method is all local search algorithm, and these methods can not ensure to obtain optimum solution, but the processing time is relatively less.These methods mainly comprise greedy algorithm and heuristic search.In addition, there is few quantifier elimination to adopt full search algorithm, optimum solution can be obtained to the problem of certain scale.

But, in actual test job, if the limited time of implementation of test cases, be difficult to perform completely under limited resources with the test use cases that conventional combination method of testing generates.At this moment, tester can wish first to perform those critical test cases, under limited resource constraint, carry out order according to the priority of test case in other words and perform them, even if make test midway stop, the test case that importance is high is also performed, improve testing efficiency with this, ensure defects detection rate.Otherwise, the problem such as even if the very large cost of flower finds optimum test use cases and have ignored the priority of test case, be easy to when test condition is limited to cause that key parameter can not be tested, wasting manpower and material resources, defects detection rate are not passed through.Therefore, we wish that the value of each parameter treating test macro determines its priority, then obtain the test use cases of a band priority according to priority.The order concentrated according to test case is in testing tested one by one, even if fail to have surveyed all parameters or combination, also farthest can ensure the quality of software systems.

Summary of the invention

Goal of the invention: the present invention is directed to test case in existing combined test method and lack the deficiency of precedence relationship, propose a kind of Pair-wise test method based on priority, by definition priority criteria, for each value of system parameter to be measured establishes priority valve, test use cases is obtained again according to greedy algorithm and genetic algorithm, thus achieve test case and generate by priority, ensure that key parameter is tested as far as possible and arrive, improve the formation speed of test case simultaneously.

Technical scheme: a kind of Pair-wise test method based on priority, is defined by priority and forms with Test cases generation algorithm two parts.Wherein: the definition of priority comprises the calculating of priority valve and preferential built-up pattern, and priority valve refers to the priority valve of certain value being gone out certain parameter according to priority key element by the formulae discovery that the present invention proposes; Preferential built-up pattern refers to ordered cover matrix, i.e. the mathematical model of test use cases of the presently claimed invention.Test cases generation algorithm generates orderly test use cases according to the priority valve of each value of parameter to be measured.

Described priority valve calculates influence factor in table 1.The each value of the present invention to parameter encloses numerical value between one 0 to 1 as its weights, and these weights are used for representing its precedence information, and its priority of the larger expression of weights is higher.The standard of weight setting is dependent on code coverage, test case executory cost tolerance, the code domain of amendment recently, amendment frequency, user's usage frequency, span etc.

Table 1 priority valve influence factor

Each factor occupies certain ratio in priority valve, and we calculate the priority valve of certain parameter value according to formula 1:

$w=c\×{\mathrm{\η}}_1+\frac{p_{\max }-p}{p_{\max }-p_{\min }}\×{\mathrm{\η}}_2+r\×{\mathrm{\η}}_3+\frac{m*}{m_{\max }}\×{\mathrm{\η}}_4+\frac{u*}{u_{\max }}\×{\mathrm{\η}}_5+\frac{v*}{v_{\max }}\×{\mathrm{\η}}_6$ Formula 1

Wherein, η ₁to η ₆represent the ratio of each factor in total priority valve respectively, be the decimal between 0 to 1, and η ₁+ η ₂+ η ₃+ η ₄+ η ₅+ η ₆=1; W represents the priority valve of certain parameter value; C represents code coverage, is a decimal in 0 to 1; P represents the testing cost of this test case, p _maxrepresent the maximum cost in set of uses case, p _minrepresent minimum cost; R represents that code is from being modified to current time measure, and it meets formula 2 below; M represents amendment frequency, m ^*represent this parameter actual modification number of times, m _maxthe amendment number of times of the maximum parameter of number of times is revised in expression system; U represents user's usage frequency, u ^*represent the discreet value of the actual access times of this parameter, u _maxthe discreet value of the parameter access times that access times are maximum in expression system; V represents span, v ^*represent the actual value number of this parameter, v _maxthe value number of the parameter that in expression system, value number is maximum.

$r=e^{-\frac{t}{\mathrm{\α}}}$ Formula 2

Wherein, t is that α is constant, and this formula meets forgetting law curve, and its metric of the code namely just revised is 1 to the maximum, along with passage of time diminishes gradually from amendment duration the last time.

After giving weights to each value of each parameter, the weights of any two tuples are that the weights of two values in two tuples are long-pending; The weights of a test case are the weights sum of two tuples occurred first that it covers.

Described preferential built-up pattern, i.e. ordered cover matrix, it is defined as: the ordered cover matrix meeting pair-wise combination coverage criteria, be one and cover array, and it meets:

(1) order of test case is according to priority worth descending sort;

(2) to any top n of this test set, their weights summation is maximum as much as possible, and namely can not find another and cover array, the weights summation of its top n test case is larger.

Described pair-wise combination coverage criteria is defined as: suppose test use cases TS _{m × k}the matrix of the capable k row of m, if two tuple-sets that in matrix, the i-th row and jth row are formed all comprise set V _i× V _j(in matrix, the i-th list shows i-th parameter, V to middle all elements _irepresent all value set of i-th parameter, V _i× V _jtwo tuple-sets of all valued combinations of expression i-th and a jth parameter), then claim TS to meet pair-wise combination coverage criteria.

The Greedy strategy of described greedy algorithm is defined as: the value choosing a parameter, the compound maximum weight of all two not capped tuples that it and fixing parameter are formed.

The coded system of described genetic algorithm is defined as: adopt binary coding mode, the possible value number of a parameter f is t, if 2n-1 < is t≤2n, so the coding figure place of this parameter is n.As following genetic algorithm encoding, be provided with a system and be made up of 4 parameters: (f1, f2, f3, f4), parameter value number is followed successively by 2, and 2,3,4; Can adopt length be 6 binary coding representation: f1 takies a b0; F2 takies a b1; F3 takies b2 and b3; F4 takies b4 and b5.For f3, front 3 codings 00,01,10 are respectively v3,0, v3,1, v3, and 2; Coding 11 can choose that of maximum weight in them.If coded representation number is m, parameter value number is n, and m > n, then the parameter value of m-n maximum weight before m-n coded representation after.

Genetic algorithm encoding

The fitness function of described genetic algorithm is defined as: using the compound weights of test case as fitness, i.e. the weights sum of two tuples occurred first of its covering, fitness function is for calculating this compound weights.

The single-point intersection of described genetic algorithm is defined as: two coded sequences are pointed out disconnection same, and the partial intersection splicing that their are disconnected.

Based on the Pair-wise test method of priority, comprise the steps:

Step 1, according to priority computing formula, calculates the priority valve of each value of each parameter, obtains two tuples and weights thereof that are made up of the value of all parameters, and they is put into two not capped tuple-set Uncover;

Step 2, according to greedy algorithm, concentrates from Uncover and tries to achieve M candidate's test case according to the priority valve situation of parameter value;

Step 3, according to genetic algorithm, encodes step 2 gained candidate test case and carries out evolutional operation, when genetic algorithm stops, choosing optimum individual and adds test case and concentrate, and concentrates from Uncover and leave out two capped tuples;

If the empty and test resource of Uncover collection still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4, according to the priority of gained test use cases, tests one by one to the parameter in system.

Described based in the Pair-wise test method of priority, the concrete implementation step of step 2 is as follows:

Step 2-1, concentrates from Uncover and chooses two large tuple t of M before weights _i(1≤i≤M if Uncover concentrates two tuple numbers individual less than M, then all chooses);

Step 2-2, according to two tuple t _idetermine candidate test case test _ithe value of two parameters in (i definition is identical with step 2-1);

Step 2-3, the loose parameter remaining to M test case, in order successively according to Greedy strategy determination value, finally obtains test _i, end step 2.

Described based in the Pair-wise test method of priority, the concrete implementation step of step 3 is as follows:

Step 3-1, encodes to M the test case that step 2 obtains;

Step 3-2, utilizes fitness function to obtain the fitness of these test cases;

If evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _maxthe individuality that the fitness of ratio is higher, chooses rear P simultaneously _minthe individuality that the fitness of ratio is lower participates in follow-on evolutionary process;

Step 3-4, the individuality chosen by step 3-3 carries out single-point intersection;

Step 3-5, does scale-of-two inversion operation to certain position in sequence to the individuality of step 3-4 gained at random by probability P m; Turn to step 3-2;

Step 3-6, the individuality choosing fitness optimum adds test case and concentrates, and concentrates from Uncover and leave out two capped tuples, end step 3.

Beneficial effect: compared with prior art, the Pair-wise test method based on priority provided by the invention, adopts the definition of priority, make gained test case can in limited resources situation test macro key parameter effectively; Genetic algorithm is added by greedy algorithm, can the generation of accelerated test use-case, reduce Test cases technology cost.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the embodiment of the present invention;

Fig. 2 is the process flow diagram of greedy algorithm in the embodiment of the present invention;

Fig. 3 is the process flow diagram of genetic algorithm in the embodiment of the present invention.

Embodiment

Below in conjunction with specific embodiment, illustrate the present invention further, these embodiments should be understood only be not used in for illustration of the present invention and limit the scope of the invention, after having read the present invention, the amendment of those skilled in the art to the various equivalent form of value of the present invention has all fallen within the application's claims limited range.

As shown in Figure 1, the Pair-wise test method based on priority that the present embodiment proposes is divided into four steps:

Step 1, sets up priority model.Being defined in summary of the invention of priority provides.For a system, first need each value of each parameter to carry out the assessment of preferential influence factor; Then the priority valve of each parameter value is obtained according to the priority valve computing formula in invention; Then the combination of each parameter value listed and calculate the priority valve of these two tuples.Provide priority model below and concrete operations be set:

If a system has A, B, C tri-parameters, wherein, parameter A has 4 values (A1, A2, A3, A4), and parameter B has 2 values (B1, B2), and parameter C has 3 values (C1, C2, C3).The value of parameter and preferential influence factor value are as table 1:

The value of table 1 parameter and preferential influence factor value

Can calculate the priority valve of each parameter value according to formula proposed by the invention, formula is as follows:

$w=c\&times;{\mathrm{\&eta;}}_1+\frac{p_{\max }-p}{p_{\max }-p_{\min }}\&times;{\mathrm{\&eta;}}_2+r\&times;{\mathrm{\&eta;}}_3+\frac{m*}{m_{\max }}\&times;{\mathrm{\&eta;}}_4+\frac{u*}{u_{\max }}\&times;{\mathrm{\&eta;}}_5+\frac{v*}{v_{\max }}\&times;{\mathrm{\&eta;}}_6$

Here we are to η 1 to η 6 assignment respectively, η 1=0.2, η 2=0.2, η 3=0.15, η 4=0.15, η 5=0.15, η 6=0.15.Here each ratio can be adjusted by actual conditions.The data of upper table are substituted into the priority valve that formula can calculate each parameter value.The priority valve situation calculated is as table 2:

The priority valve of table 2 parameter value

After obtaining the priority valve of each parameter value, the combination of parameter value and its weights are obtained, the weights after combination are the product of two value priority valves, as table 3:

The combination of table 3 parameter value and priority valve thereof

Owing to there is estimation in some sampling process, so final priority valve only need retain the precedence case that two-decimal can reflect combination, so done approximate processing to all priority valves calculated in table 3.

After priority model is set up, to two not capped tuple-set Uncover initialization, all binary combination are put into wherein.

Step 2, according to greedy algorithm, concentrate from Uncover and try to achieve M candidate's test case according to the priority valve situation of parameter value, as shown in Figure 2, concrete steps are as follows for its algorithm flow:

Step 2-1, concentrates from Uncover and chooses two large tuple t of M before weights _i(1≤i≤M if Uncover concentrates two tuple numbers individual less than M, then all chooses);

Carry out descending sort to Uncover collection, the set obtained is as table 4:

Table 4 is according to priority worth the Uncover set of descending

Here we establish M=9, also can adjust the value of M according to actual conditions.Namely concentrate two tuples choosing front 8 maximum weight from Uncover, be respectively: A4C1, A4C2, B2C1, A1C1, A4C3, B1C1, B2C2, A1C2, B1C2.

Step 2-2, according to two tuple t _idetermine candidate test case test _ithe value of two parameters in (i definition is identical with step 2-1), the value namely in these 8 two tuples is fixed, and they are as the part determined in 8 candidate's test cases;

Step 2-3, the loose parameter remaining to M test case, in order successively according to Greedy strategy determination value, finally obtains test _i.

Greedy strategy of the present invention is the value choosing a parameter, the compound maximum weight of all two not capped tuples that it and fixing parameter are formed.Such as, for A4C1, lack the value of parameter B, therefore it can combine with B1 or B2.If with B1 combines, then its compound weights are:

Weights=the 0.20+0.32+0.27=0.79 of the weights+B1C1 of the weights+A4C1 of A4B1

If with B2 combines, then its compound weights are:

Weights=the 0.22+0.32+0.29=0.83 of the weights+B2C1 of the weights+A4C1 of A4B2

Can find out, if combine with B2, then compound weights are larger, therefore select B2 and its combination, produce a candidate test case A4B2C1.Such operation is performed to each two tuple and can obtain whole candidate's test case.

Step 3, according to genetic algorithm, encodes step 2 gained candidate test case and carries out evolutional operation, when genetic algorithm stops, choosing optimum individual and add test case and concentrate, and concentrate from Uncover and leave out two capped tuples, genetic algorithm flow process is as Fig. 3, and concrete steps are as follows;

Step 3-1, encodes to 9 test cases that step 2 obtains;

Coding needs to determine binary figure place according to parameter value number.As parameter A have 4 values, parameter B has 2 values, parameter C has 3 values, then parameter A 2 bit representations, parameter B 1 bit representation, parameter C 2 bit representations, corresponding coding and value relation as shown in table 5:

Table 5 coding and value relation

Parameter C only has 3 values, coding can represent 4, therefore that value that last coded representation priority valve is maximum.

11100 are encoded to for the such test case of A4B2C1; Such as, as long as corresponding value can be found according to table 5,10101 corresponding A 3B2C2 for a coding.

Step 3-2, utilizes fitness function to obtain the fitness of these test cases, here we using the compound weights of a test case as fitness;

If individual evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _maxthe individuality that the fitness of ratio is higher, chooses rear P simultaneously _minthe individuality that the fitness of ratio is lower participates in follow-on evolutionary process; Here we can choose P _max, P _minbe respectively 60% and 20%.

Step 3-4, the individuality chosen by step 3-3 carries out single-point intersection; Stochastic choice point of crossing, carry out when intersecting, two before or after this point individual part-structures exchange, and generate two new individual.Such as:

Individual A:001 ↑ 11 → 00101 (new individual A ')

Individual B:100 ↑ 01 → 10011 (new individual B ')

Step 3-5, does scale-of-two inversion operation to certain position in sequence to the individuality of step 3-4 gained at random by probability P m.Pm is an experience amount, adjusts according to actual conditions; Turn to step 3-2;

Step 3-6, the individuality choosing fitness optimum adds test case and concentrates, and concentrates from Uncover and leave out two capped tuples.

Such as we are A4B2C1 by the optimum individual that first round genetic algorithm obtains, then need all two tuples that it is comprised to concentrate from Uncover and delete, two tuples that should delete are A4B2, A4C1, B2C1.

If the empty and test resource of Uncover collection still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4, according to the test use cases that the present invention obtains, tests one by one to the parameter in system.Due to concentrate in test case more forward test case its to meet weights larger, therefore can guarantee that system core parameter is tested, by defects detection rate according to this test case sequential testing.

Claims (1)

1. the Pair-wise test method based on priority, it is characterized in that: comprise models of priority and set up and Test cases technology step, wherein: described models of priority establishment step comprises the definition of parameter value priority, the calculating of priority valve, the priority valve of parameter value two tuple and the priority valve of test case; Described Test cases technology step comprises by greedy algorithm generation candidate's test case and utilizes genetic algorithm to generate optimum test case two parts;

Priority-sensitive factor comprises: code coverage, cost metric, the time measure apart from last change, amendment frequency, user's usage frequency, span;

Priority computing formula is as formula 1:

formula 1

Wherein, η ₁to η ₆represent the ratio of each influence factor in total priority valve respectively, be the decimal between 0 to 1, and η ₁+ η ₂+ η ₃+ η ₄+ η ₅+ η ₆=1; W represents the priority valve of certain parameter value; C represents code coverage, is a decimal in 0 to 1; P represents the testing cost of this test case, p _maxrepresent the maximum cost in set of uses case, p _minrepresent minimum cost; R represents that code is from being modified to current time measure, and it meets formula 2 below; M represents amendment frequency, and m* represents this parameter actual modification number of times, m _maxthe amendment number of times of the maximum parameter of number of times is revised in expression system; U represents user's usage frequency, and u* represents the discreet value of the actual access times of this parameter, u _maxthe discreet value of the parameter access times that access times are maximum in expression system; V represents span, and v* represents the actual value number of this parameter, v _maxthe value number of the parameter that in expression system, value number is maximum;

formula 2

Wherein, t is that α is constant, and this formula meets forgetting law curve, and its metric of the code namely just revised is 1 to the maximum, along with passage of time diminishes gradually from amendment duration the last time;

The priority valve of described parameter value two tuple is that the weights of two values in two tuples are long-pending;

The priority valve of described test case is the weights sum of two tuples occurred first that it covers;

The Greedy strategy of described greedy algorithm is defined as: the value choosing a parameter, the compound maximum weight of all two not capped tuples that it and fixing parameter are formed;

The coded system of described genetic algorithm is defined as: adopt binary coding mode, the possible value number of a parameter f is t, if 2n-1 < is t≤2n, so the coding figure place of this parameter is n; If coded representation number is m, parameter value number is n, and m > n, then the parameter value of m-n maximum weight before m-n coded representation after;

The fitness function of described genetic algorithm is defined as: using the compound weights of test case as fitness, i.e. the weights sum of two tuples occurred first of its covering, and fitness function is for calculating this compound weights;

The single-point intersection of described genetic algorithm is defined as: disconnected in same point by two coded sequences, and the partial intersection splicing that their are disconnected;

Comprise the steps:

Step 1, according to priority computing formula, calculates the priority valve of each value of each parameter, obtains two tuples and weights thereof that are made up of the value of all parameters, and they is put into two not capped tuple-set Uncover;

Step 2, according to greedy algorithm, concentrates from Uncover and tries to achieve M candidate's test case according to the priority valve situation of parameter value;

Step 3, according to genetic algorithm, encodes step 2 gained candidate test case and carries out evolutional operation, when genetic algorithm stops, choosing optimum individual and adds test case and concentrate, and concentrates from Uncover and leave out two capped tuples;

If the empty and test resource of Uncover collection still allows to test more test case, then turn to step 2; Otherwise turn to step 4;

Step 4, according to the priority of gained test use cases, tests one by one to the parameter in system;

The concrete implementation step of described step 2 is as follows:

Step 2-1, concentrates from Uncover and chooses two large tuple ti of M before weights, wherein 1≤i≤M, if Uncover concentrates two tuple numbers individual less than M, then all choose;

Step 2-2, determines the value of two parameters in candidate test case testi according to two tuple ti, i definition is identical with step 2-1;

Step 2-3, the loose parameter remaining to M test case, in order successively according to Greedy strategy determination value, finally obtains testi, end step 2;

The concrete implementation step of described step 3 is as follows:

Step 3-1, encodes to M the test case that step 2 obtains;

Step 3-2, utilizes fitness function to obtain the fitness of described test case;

If evolution number of times is enough, then turn to step 3-6; Otherwise turn to step 3-3;

Step 3-3, P before selecting _maxthe individuality that the fitness of ratio is higher, chooses rear P simultaneously _minthe individuality that the fitness of ratio is lower participates in follow-on evolutionary process;

Step 3-4, the individuality chosen by step 3-3 carries out single-point intersection;

Step 3-5, does scale-of-two inversion operation to certain position in sequence to the individuality of step 3-4 gained at random by probability P m; Turn to step 3-2;

Step 3-6, the individuality choosing fitness optimum adds test case and concentrates, and concentrates from Uncover and leave out two capped tuples, end step 3.