CN103631714A - Method for generating minimum combination testing cases based on matrix multiplicity - Google Patents

Abstract

The invention discloses a method for generating minimum combination testing cases based on matrix multiplicity. The method includes the steps that firstly, an input matrix is generated according to the number of factors and the number of values of each factor; secondly, an initial solution matrix is generated through the multiplicity comparison method; thirdly, whether the generated solution matrix can cover all factor combinations or not is verified, and factor combinations which are not covered are recorded; fourthly, the factor combinations which are not covered are covered based on the conception of the greedy algorithm to obtain the final solution matrix, wherein each line in the solution matrix is one testing case. The method has the advantage that the number of the generated testing cases is small under the condition that all the factor combinations are covered; through comparison experiments between the method and existing normally-used combination testing tools like an ACTS and a PICT, the effectiveness of the method and the advantage that the number of the generated combination testing cases is small are verified.

Description

Minimum combination method for generating test case based on matrix multiplicity

Technical field

The invention belongs to the combined test use-case generation technique field in software test, relate to a kind of minimum combination method for generating test case based on matrix multiplicity.

Background technology

Conventionally software systems are complicated flogic systems, and its normal operation is subject to multifactorial impact, and these factors comprise system configuration, internal event, outside input and running environment etc., can abstract representation the input factor that is software systems.The interaction of these factors can cause the inefficacy of software conventionally, in actual software test, can to software systems, test fully by testing all combinations of these systematic parameters.Have n input parameter (factor) for one, the value number of each factor is v ₁, v ₂..., v _nsoftware systems, according to these software systems needs of input parameter combination Complete test

individual test case, for general test macro, this combination is a very huge numeral.For use test use-case as few as possible carrys out effectively more early to detect software error, people have proposed the method for testing software covering based on combination, are called for short combined test (Combination Testing).At international esbablished corporations such as industry member ，IBM, Microsoft and Bell laboratories, all developed corresponding combined test instrument, these combined test instruments, aspect test case generating algorithm, generally adopt the greedy algorithm of speed.The thought of greedy algorithm is from empty matrix, and progressively or column by column extended matrix, until all t combination is all capped.According to the difference of extended mode, can be divided into One-Dimensional Extended and two-dimensional expansion two classes.The algorithm frame of One-Dimensional Extended is realized by business test case Core Generator AETG at first, the Greedy strategy of AETG is non-deterministic, so repeatedly move the test case the possibility of result difference that AETG obtains, thus be difficult to guarantee that the combined test set of uses case generating minimizes, and unstable result.The method of the similar AETG of method that the instrument PICT of Microsoft's exploitation adopts, but the Greedy strategy of PICT is deterministic, repeatedly moves coming to the same thing of PICT, but cannot guarantee that equally the combined test set of uses case of its generation minimizes.In the mode of two-dimensional expansion, the people such as Lei have proposed IPO (in-parameter-order) algorithm, this algorithm is mainly for pairwise testing, can only produce at first paired covering array, the people such as Lei expanded this algorithm afterwards, had obtained a kind of algorithm IPOG that can produce any t combined test use-case.Although IPOG algorithm is greedy algorithm, it has adopted the method for exponential complexity to select minimum row to carry out extends perpendicular, therefore still need to spend the plenty of time when processing extensive example.

In order further to reduce combined test use-case quantity, save software test cost.We change into matrix form the software input space, then by the multiplicity in the ranks of test case in judgment matrix, generate the minimum dematrix that covers, and finally generate minimum combined test set of uses case.

Summary of the invention

Although the greedy algorithm efficiency such as PICT recited above, IPOG are higher, the test case quantity generating is many.The present invention is directed to a fairly large number of shortcoming of test case that current method generates, a kind of minimum combination method for generating test case based on matrix multiplicity has been proposed, guaranteeing in the nondecreasing situation of coverage rate that combination covers, make the test case number of generation still less, thereby improve the efficiency of software test, reduce the cost of software test.And the result of the experimental result of this method and existing combined test instrument ACTS and PICT is compared, verified the validity of put forward the methods and the few advantage of test case number of generation.

Design of the present invention is as follows:

Step 1, generates input matrix A according to factor number and each factor value number;

Step 2, adopts the method for heavier multiplicity to generate preliminary dematrix B;

Step 3, whether the dematrix B that checking generates can cover all factor combinations, will not have chlamydate factor combination to be recorded in Matrix C;

Step 4, adopts the thought of greedy algorithm to cover Matrix C, obtains final dematrix B.

The concrete steps of above-mentioned steps 1 are as follows:

Step 1.1, determines the columns of input matrix A and the maximal value of each factor according to factor number;

Step 1.2, is set to initial value by each factor, and this line is joined in input matrix A;

Step 1.3, the lowest order of factor adds 1, if be greater than this maximal value, forward carry;

Step 1.4, if most significant digit is less than or equal to the maximal value of most significant digit, joins a line just having generated in output matrix A, then forwards step 1.3 to, if most significant digit is greater than the maximal value of most significant digit, and the input matrix A that output obtains, algorithm finishes.

In above-mentioned steps 1.1-step 1.4, the initial value of each factor and span are made following agreement, the initial value of the 1st factor is 100, if the value number of first factor is 3, the span of the 1st factor is 100～102 integer, and maximal value is that the initial value of 102, the second factors is 200, the 3rd initial value is 300, by that analogy.

The concrete steps of above-mentioned steps 2 are as follows:

Step 2.1, initialization dematrix B, joins the first row of the input matrix A obtaining in step 1 in dematrix B;

Step 2.2, is made as the first row of input matrix A when pre-treatment row;

Step 2.3, compares the every a line in pre-treatment row and dematrix B one by one, if multiplicity all meets the demands, current line is joined in dematrix B, then from input matrix A, deletes current line;

Step 2.4, if also there is untreated row in input matrix A, is made as this row when pre-treatment row, forwards step 2.3 to, otherwise output dematrix B, algorithm finishes.

The concrete steps of above-mentioned steps 3 are as follows:

Step 3.1, chooses n factor according to factor coverage n, if the combination of this n factor was not selected, forwards step 3.2 to, otherwise forwards step 3.4 to;

Step 3.2, generates the value of this n factor, if the combination of the value of this n factor was not generated, forwards step 3.3 to, otherwise forwards step 3.1 to;

Step 3.3, whether the combination of the value of n the factor that judgement generates is covered by dematrix B, if not capped, this factor combination is joined in Matrix C, forwards step 3.2 to;

Step 3.4, output matrix C, algorithm finishes.

The concrete steps of above-mentioned steps 4 are as follows:

Step 4.1, the every a line in calculating input matrix A can cover the number of the row in Matrix C, and note, in array A, is marked under the row of covering in array B;

Step 4.2, if not capped line number is greater than 0 in Matrix C, forwards step 4.3 to, otherwise forwards step 4.5 to;

Step 4.3, chooses one maximum in array A, this line is added in dematrix B, then according to the record in array B, these row states in Matrix C are made as and are covered;

Step 4.4, the information of renewal array A and array B, forwards step 4.2 to;

Step 4.5, output dematrix B, algorithm finishes.

Accompanying drawing explanation

Fig. 1 is the minimum combination method for generating test case process flow diagram based on matrix multiplicity.

Fig. 2 is the process flow diagram that generates input matrix A.

Fig. 3 is the process flow diagram that generates preliminary dematrix B.

Fig. 4 is the process flow diagram of generator matrix C.

Fig. 5 is the process flow diagram that generates final dematrix B.

Fig. 6 is scale F=3 ¹⁰time this method and the IPOG of ACTS instrument, the test case of the IPOG-D of ACTS instrument, PICT instrument generates quantity comparison diagram.

Fig. 7 is scale F=4 ⁸time this method and the IPOG of ACTS instrument, the test case of the IPOG-D of ACTS instrument, PICT instrument generates quantity comparison diagram.

Fig. 8 is scale F=2 ¹²time this method and PICT instrument test case generate quantity comparison diagram.

Fig. 9 is scale F=2 ¹³time this method and PICT instrument test case generate quantity comparison diagram.

Embodiment

In order more clearly to understand the technology contents of the minimum combination method for generating test case that the present invention is based on matrix multiplicity, below in conjunction with accompanying drawing and a case study on implementation, the invention will be further described, it should be noted that, the case study on implementation providing of describing is intended to be convenient to the understanding of the present invention, and to it without any limiting requirement.

As shown in Figure 1, the first step is inputted factor number and each factor value number to the process flow diagram of the minimum combination method for generating test case based on matrix multiplicity that the present invention provides; Second step generates input matrix A, and the 3rd step adopts the method for heavier multiplicity to generate preliminary dematrix B; Whether the dematrix B that the 4th step card generates can cover all factor combinations, will not have chlamydate factor combination to be recorded in Matrix C; The 5th step adopts the thought of greedy algorithm to cover Matrix C, obtains final dematrix B.

First, provide the several concept definitions that arrive involved in the present invention as follows.

Definition 1(test use cases): the total n of input factor that supposes software to be tested is individual, forms input space S set={ f ₁, f ₂..., f _n, factor f wherein _icomprise a _iindividual value, this factor value set is V _i(1≤i≤n), claims a n tuple t=(v ₁, v ₂..., v _n) (v ₁∈ V ₁, v ₂∈ V ₂..., v _n∈ V _n) be a test case of software to be tested, a plurality of n tuple t are just a test use cases of software to be tested, this test use cases can change into matrix form.

The minimum N factor combination of definition 2(covers matrix): the input space of establishing software systems can be expressed as m * n matrix, note A=(a _ij) _{m * n}.The factor f of software to be tested is shown in its j list _j, this column element is taken from set V _j(j=1,2 ..., n), i.e. a _ij∈ V _j.If the N that appoints of A is listed as, i.e. i ₁, i ₂..., i _nrow all meet: V _i1, V _i2..., V _inall N dimension combinations of middle symbol are all listed as in the orderly group of formed N unit and at least occur once at this N, claim that A is that a N factor combination covers matrix.Every a line of A is exactly a test case, and m is the number of test case.If m is the minimum positive integer that guarantees that above-mentioned condition is set up, claim that A is that minimum N factor combination covers matrix.

Definition 3(dematrix): dematrix is m * n matrix, note B=(b _ij) m * n.Wherein m is test case number, only contains a line (individual) test case of first value composition of each factor in initial B.It is that minimum two factors combinations cover matrixes that two factors cover dematrixs, and it is that minimum three factors combinations cover matrixes that three factors cover dematrixs, and N factor to cover dematrix be that minimum N factor combination covers matrix.

Definition 4(multiplicity): establish t ₁=(v ₁, v ₂..., v _n) and t ₂=(s ₁, s ₂..., s _n) be two test cases of software to be tested.If t ₁and t ₂on h position, respective value is identical, and the multiplicity that claims two test cases is h.

Definition 5(matrix solution): establish t _i=(v ₁, v ₂..., v _n), and if t _ithe multiplicity that covers arbitrary row test case in dematrix B with two factors is less than or equal to 1, t _iit is the solution that two factors cover dematrix B.If t _ithe multiplicity h that covers arbitrary row test case in dematrix B with three factors is less than or equal to 2, t _iit is the solution that three factors cover dematrix B.In like manner, if t _ithe multiplicity h that covers arbitrary row test case in dematrix B with N factor is less than or equal to N-1, t _ia solution for N factor covering dematrix B.

The software parameter factor number in the input of the first step of take is below 5, and each factor value number is 2 for case study on implementation, and the implementation process of the inventive method is described.

With reference to Fig. 2, the process that generates input matrix A is as follows:

Step 201 is determined columns and each factor maximal value of input matrix A, and in this example because factor number is 5, each factor value number is 2, so the columns of input matrix A is 5 row, the maximal value of each factor is respectively 101,201,301,401,501; Step 202 adds initial row in input matrix A, and first each factor is set to initial value, is 100,200,300,400,500 here, then this row is joined in input matrix A; Step 203 factor lowest order adds 1, and in about set matrix, the Far Left of every row is most significant digit, and rightmost is lowest order, after lowest order adds 1, becomes 100,200,300,400,501 here; Step 204 judges whether present bit is greater than this maximal value, forwards step 205 if be not more than to, otherwise forwards step 206 to, and present bit 501 is not more than this maximal value 501 herein, forwards step 205 to; Step 205 joins current line in matrix A, then forwards step 203 to, and here by 100,200,300,400,501 these row add in matrix A; Step 206 is carry forward, by current location, is first initial value, then last position is added to 1, and is present bit by last position, when factor row becomes 100,200, within 300,400,502 o'clock, just needs carry forward, becomes 100,200,300,401,500; Step 207 judges whether most significant digit is greater than most significant digit maximal value, if be greater than, algorithm finishes otherwise forwards step 204 to, becomes 102,200,300,400 here after factor carry, and within 500 o'clock, most significant digit has just been greater than most significant digit maximal value, and algorithm finishes.

With reference to Fig. 3, the process that generates preliminary dematrix B is as follows:

Step 301 initialization dematrix B, joins the first row of input matrix A in matrix B; Step 302 is made as current line by the first row in input matrix A; Step 303 is the heavier multiplicity one by one of the row in pre-treatment row and dematrix B, and multiplicity refers to the identical number of factor value in two row of comparison here, and for example 100,200,300,400,500 and 100,200,300,401,501 multiplicity is 3; Step 304 judges whether multiplicity all meets the demands, and meets and forwards step 305 to, does not meet and forwards step 306 to, the multiplicity here meets the demands and refers to: if ask the combination of n factor to cover, multiplicity must be less than n, take n=3 here as example, in matrix A 100,200,301,401,501 with B in unique row 100,200,300,400,500 multiplicities are less than 3, meet the demands; Step 305 joins current line in dematrix B, and in A, deletes current line; Also whether step 306 judge in A and have and forward step 307 to, otherwise algorithm to finish promising processing row; Step 307 is made as current line by the next one in A for processing row.

With reference to Fig. 4, the process of generator matrix C is as follows:

Step 401 judges whether to exist the combination of n factor not to be selected, if forward step 402 to, otherwise algorithm finishes, here the value of n cover with desired n factor combination in the value of n identical, for example in present example, require 3 factors to cover, n value is 3 here; Step 402 is chosen this n the factor not being selected; Step 403 judges whether to exist the combination of the value of this n factor not to be selected, if forward step 404 to, otherwise forwards step 401 to; The combination that the value of this n of step 404 generation factor was not selected; Whether the combination of the value of n the factor that step 405 judgement generates is covered by matrix B, if forward step 403 to, otherwise forwards step 406 to; Step 406 adds Matrix C by the combination of just having found.

With reference to Fig. 5, the process that generates final dematrix B is as follows:

Step 501 is calculated array A and B, notes down the number that every a line in input matrix A can cover the row in Matrix C in A, notes down the subscript of these row, i.e. the position of these row in C in B; In step 502 judgment matrix C, whether there is unlapped row, if existed, forward step 503 to, otherwise algorithm finishes; Step 503 finds peaked position in A, finds which in A can cover row maximum in C, then this line in A is joined in dematrix B; Step 504 is made as the corresponding line state in C and covers according to B data; Step 505 is upgraded array A and B.

For the validity of checking this method, collected these two combined test instruments of ACTS and PICT, the IPOG-D algorithm of the IPOG algorithm of this method and ACTS, ACTS and PICT algorithm have been carried out to contrast experiment.First the scale of set of uses case has selected F=3 ¹⁰and F=4 ⁸, from 2 factor combinations, covering until 6 factor combinations cover respectively, as shown in Table 1 and Table 2, corresponding graphical result is as shown in Figure 6 and Figure 7 for experimental result.Result demonstration, the test case quantity that this method generates is compared obvious minimizing with other several methods, but has reached same combination level of coverage, has verified the few advantage of this method generating test use case quantity.Then the scale selection of set of uses case F=2 ¹²and F=2 ¹³carry out high factor covering, from 2 factor combinations, cover until total factor combination covers (being that 12 factor combinations cover and the combination of 13 factors covers) here respectively, because IPOG and IPOG-D do not support 6 factors to cover above, here compare with the good PICT of effect in IPOG, IPOG-D and tri-kinds of algorithms of PICT, as shown in Table 3 and Table 4, corresponding graphical result as shown in Figure 8 and Figure 9 for experimental result.Result demonstration, this method not only can support high factor to cover, and still keeps the advantage that generating test use case quantity is few.

The scale F=3 of table 1 set of uses case ¹⁰time each algorithm test case number comparison of generating

N factor	This method	ACTS(IPOG)	ACTS(IPOG-D)	PICT
					2	15	15	21	18
3	60	66	71	65
					4	208	233	278	231
5	660	761	1118	735
					6	1966	2262	2834	2164

The scale F=4 of table 2 set of uses case ⁸time each algorithm test case number comparison of generating

N factor	This method	ACTS(IPOG)	ACTS(IPOG-D)	PICT
					2	28	30	32	28
3	112	121	124	139
					4	496	670	720	600
5	1792	2375	2816	2361
					6	7330	9824	10848	8330

The scale F=2 of table 3 set of uses case ¹²time each algorithm test case number comparison of generating

N factor	This method	PICT
				2	9	9
3	20	19
			4	47	48
5	85	103
			6	203	222
7	256	428
			8	742	786
9	1280	1371
			10	2324	2176
11	2048	2717
			12	4096	4096

[0065]the scale F=2 of table 4 set of uses case ¹³time each algorithm test case number comparison of generating

N factor	This method	PICT
				2	9	9
3	22	21
			4	47	52
5	108	113
			6	229	244
7	256	485
			8	869	925
9	1651	1670
			10	3045	2821
11	4496	4477
			12	4096	6106
13	8192	8192

Claims (5)

1. the minimum combination method for generating test case based on matrix multiplicity, is characterized in that, comprises the following steps:

Step 1, generates input matrix A according to parameter factor number and each factor value number;

Step 2, adopts the method for heavier multiplicity to generate preliminary dematrix B;

Step 3, whether the dematrix B that checking generates can cover all parameter factor combinations, will not have chlamydate factor combination to be recorded in Matrix C;

Step 4, adopts the thought of greedy algorithm to cover Matrix C, obtains final dematrix B.

2. method according to claim 1, is characterized in that: the concrete steps of described step 1 are as follows:

Step 1.1, determines the columns of input matrix A and the maximal value of each factor according to factor number;

Step 1.2, is set to initial value by each factor, and this line is joined in input matrix A;

Step 1.3, the lowest order of factor adds 1, if be greater than this maximal value, forward carry;

Step 1.4, if most significant digit is less than or equal to the maximal value of most significant digit, joins a line just having generated in output matrix A, then forwards step 1.3 to, if most significant digit is greater than the maximal value of most significant digit, and the input matrix A that output obtains, algorithm finishes.

3. method according to claim 1, is characterized in that: the concrete steps of described step 2 are as follows:

Step 2.1, initialization dematrix B, joins the first row of the input matrix A obtaining in step 1 in dematrix B;

Step 2.2, is made as the first row of input matrix A when pre-treatment row;

Step 2.3, compares the every a line in pre-treatment row and dematrix B one by one, if multiplicity all meets the demands, current line is joined in dematrix B, then from input matrix A, deletes current line;

Step 2.4, if also there is untreated row in input matrix A, is made as this row when pre-treatment row, forwards step 2.3 to, otherwise output dematrix B, algorithm finishes.

4. method according to claim 1, is characterized in that: the concrete steps of described step 3 are as follows:

Step 3.1, chooses n factor according to factor coverage n, if the combination of this n factor was not selected, forwards step 3.2 to, otherwise forwards step 3.4 to;

Step 3.2, generates the value of this n factor, if the combination of the value of this n factor was not generated, forwards step 3.3 to, otherwise forwards step 3.1 to;

Step 3.3, whether the combination of the value of n the factor that judgement generates is covered by dematrix B, if not capped, this factor combination is joined in Matrix C, forwards step 3.2 to;

Step 3.4, output matrix C, algorithm finishes.

5. method according to claim 1, is characterized in that: the concrete steps of described step 4 are as follows:

Step 4.1, the every a line in calculating input matrix A can cover the number of the row in Matrix C, and note, in array A, is marked under the row of covering in array B;

Step 4.2, if not capped line number is greater than 0 in Matrix C, forwards step 4.3 to, otherwise forwards step 4.5 to;

Step 4.3, chooses one maximum in array A, this line is added in dematrix B, then according to the record in array B, these row states in Matrix C are made as and are covered;

Step 4.4, the information of renewal array A and array B, forwards step 4.2 to;

Step 4.5, output dematrix B, algorithm finishes.

Images