CN102831065B - Statistics software defect locating method for coincidence consistency problems - Google Patents

Abstract

A statistics software defect locating method for coincidence consistency problem comprises five steps of (1) executing instrumentation on a program with defects; (2) loading a test case of the program, and running the program after being subjected to the instrumentation; (3) collecting output information of stub functions when the program is successful and fails to run; (4) analyzing the collected predicate dynamic execution information, including the steps of calculating an overlapping degree of the distribution of predicate success and failure executive spectrums, calculating standardized between-class distance of the predicate executive spectrum distribution, calculating predicate suspicious degree and sequencing the predicates; and (5) searching the predicates according to the sequence in a predicate sequence table obtained in the fourth step until the predicate associated with the defects is found. The method can effectively avoid the influence of the coincidence consistency, and is high in locating efficiency, simple and easy to realize.

Description

A kind of statistics software defect positioning method towards coincidence consistency problem

Technical field

The present invention relates to a kind of dynamic software defect positioning method, particularly relate to a kind of statistics software defect positioning method towards coincidence consistency problem, it is a kind of statistics software defect positioning method being integrated with probability distribution criterion and the calculating of standardization between class distance.The method belongs to software testing technology field.

Background technology

(the method is a kind of algorithm to statistics software defect positioning method in essence, hereinafter usage out of habit, same algorithm one word) perform the difference of composing by program element during comparison program operation success and failure, search most possible relevant with bugs program element.

In existing statistics software defect positioning method, the more representational Tarantula of being algorithm, CBI algorithm, SOBER algorithm (these titles are inventor's names of algorithm, now temporarily without Chinese).To be the people such as Harold propose Tarantula algorithm in " a kind of utilize the defect positioning method of visual information " (referring to 2002 " the 24 international soft project meeting ") literary composition, its pile pitching method selected is statement pitching pile, namely carry out pitching pile to all executable statements, statistics success and failure test case is to the coverage condition of every bar executable statement.This algorithm based on hypothesis be that then this executable statement is more suspicious if executable statement is covered by all failure testing use-cases and is not tested successfully use-case and covers.But most of statement all seldom causes defect in program, the defect location for all executable statements not only can cause the redundancy of information, but also defect location cost can be made to increase.For this problem, this. in lining, top grade people proposes CBI algorithm in " a kind of failure separation method of extendible Corpus--based Method " (referring to 2005 " the Design and implementation > meeting of american computer association < programming language ") literary composition.The pile pitching method of this algorithms selection is predicate pitching pile, carries out pitching pile to branch statement, return statement and quantity to statement, realizes defect location by collecting and process the Dynamic Execution information of predicate when program is run.Although the special information redundancy problem solved in Tarantula algorithm in lining, but do not collect the execution information (be mainly manifested in spy in lining only to consider predicate and whether be performed in a secondary program is run, do not consider that this predicate is performed several times in a secondary program) of predicate fully.The locating effect of the insufficient CBI of the causing algorithm of information need to improve.For this reason, the superfine people of Liu proposes SOBER algorithm in " program correction of Corpus--based Method: a kind of method based on test of hypothesis " (referring to 2006 " Institute of Electrical and Electronic Engineers's soft project transactions ") literary composition.This algorithm efficiently solves the insufficient problem of predicated execution information, show as it not only to have recorded predicate and whether be performed in a secondary program is run, and have recorded the number of times of predicate execution in a secondary program is run, comprise the number of times that predicate is judged as "True" and is judged as "false".These two kinds of statistical informations are also included in the middle of the calculating of the predicate defect degree of correlation, and the method introducing test of hypothesis is to calculate the suspicious degree of predicate.Experimental result shows, the locating effect of SOBER algorithm is better than CBI algorithm.But SOBER algorithm needs hypothesis predicated execution Spectral structure to be normal distribution when applying, the people such as Zhang Zhenyu once proved that the predicated execution of some program was composed and do not meet normal distribution in " non-ginseng statistical deficiency location " (referring to 2011 " system and software periodical ") literary composition, therefore proposed Wilcoxon algorithm and Mann-Whitney algorithm.This algorithm utilize the method for nonparametric hypothesis test in statistics difference that comparison predicate performs spectrum, through experimental results demonstrate, Wilcoxon algorithm and Mann-Whitney algorithm existing based on the defect location algorithm of predicate in locating effect optimum.

Although statistics software defect positioning method can obtain very high positioning precision, the positioning precision of these class methods can by the impact of input data characteristic.Research before finds, the existence of coincidence consistance (when coincidence consistance refers to that program is run, defect statement is performed, but program does not show inefficacy) problem can reduce the positioning precision of statistical deficiency localization method.Because the execution profile that coincidence makes program run successful test case is with to make program run the execution profile of failed test case close, if this part test case be included in successful test case, can perform to comparison predicate success and failure that spectral difference is different has a negative impact.Therefore, prior art mainly concentrates on and finds the program coincidence of making to run correct test case, and is removed.But first these methods are limited to how correctly to find exists the conforming test case of coincidence.Secondly, remove and there is the conforming test case of coincidence, the test data of input will be reduced, affect the application of statistical method in algorithm.

How under the prerequisite allowing coincidence consistance to exist, utilize the correlation properties of coincidence consistency problem to carry out defect location just the present invention based on consideration.

Summary of the invention

A kind of statistics software defect positioning method towards coincidence consistency problem of the present invention, its objective is: overcome the shortcoming that existing method is subject to the impact of coincidence consistency problem, from the characteristic of coincidence consistency problem itself, provide a kind of new statistics software defect positioning method.

A kind of statistics software defect positioning method towards coincidence consistency problem of the present invention, its design philosophy is: be divided into by the pitching pile predicate in program neutral predicate (with bugs without any data and control association relation, and at a distance of defect predicate far away), cause the predicate of defect (to have data or control association relation with bugs, and the predicate be positioned at before data or control association relation, also comprises defect predicate itself) and by predicate three class of defective effect.On program operation logic, performing spectrum owing to causing the success and failure of the predicate of defect can exist certain overlapping, and by the predicate of defective effect success and failure perform spectrum do not have overlapping, according to predicate success and failure perform spectrum overlapping degree difference can by cause the predicate of defect to come by defective effect predicate before.Then refinement is carried out in the sequence of calculating to the predicate causing defect by standardization between class distance, thus causes the predicate of defect to come relatively forward position preferentially checking by the most suspicious.Pile function specifically in pitching pile program returns the Dynamic Execution information (mainly referring to the evaluation error of predicate herein) of predicate.Correspondingly, working procedure collect this information.Finally, the statistical method of design is adopted to carry out interpretation of result.During concrete analysis, the mode utilizing probability distribution criterion and standardization between class distance to calculate to combine is to calculate the Defect Correlation degree of each predicate.Then, from high to low predicate is arranged according to Defect Correlation degree, thus realize the object of carrying out defect location according to predicate sequence.

More specifically, a kind of statistics software defect positioning method towards coincidence consistency problem of the present invention, its step comprises following five steps:

The first step carries out pitching pile to containing defective program.Program pitching pile is widely used in software test, and pitching pile is to obtain the multidate information of specific program node when program is run.Present patent application mainly carries out pitching pile to the branch statement in program, return statement and quantity to statement, it is achieved by inserting Boolean expression (abbreviation predicate), (in the present patent application, the Dynamic Execution information spinner of predicate will refer to the evaluation error of predicate to the Dynamic Execution information of pile function requirement outer predicate when program is run.The definition of evaluation error is: in a secondary program operation, predicate is judged as the probability of "True".Expression formula can be used represent, wherein, e _t(p _j, r _i) and e _f(p _j, r _i) difference representation program r _ipredicate p during secondary operation _jbe judged as "True" and the number of times being judged as "false".If it should be noted that, predicate is not performed in certain secondary program is run, then evaluation error value is now set to 0.5).For ease of hereinafter illustrating, use p ₁, p ₂..., p _mm pitching pile predicate in representation program;

Whole test cases of second step loading procedure, run the program being inserted with pile function.Exported and defect version (implanting the program after individual defect) output by contrast prototype version (program not containing any defect), distinguish and program is run successfully and runs failed test case.Wherein, use u representation program to run successful number of times, v representation program runs failed number of times;

3rd step collects the predicate Dynamic Execution information that when every secondary program is run, pile function exports.When program is run successfully, predicate p _j(j ∈ [1, m]) obtains u evaluation error altogether, when program is run unsuccessfully, and predicate p _jobtain v evaluation error altogether;

4th step carries out statistical analysis to all predicate Dynamic Execution information collected, and this step is divided into again four small steps:

(1) predicate p is calculated _j(j ∈ [1, m]) success and failure performs the overlapping degree of Spectral structure, uses O _jrepresent.

For predicate p _jthe execution spectrum set that assessment deviate is formed respectively when program runs success and failure, utilizes probability distribution criterion $O_j=-\ln \left[{\mathrm{\&Sigma;}}_{y_j\&Element;D_j}\sqrt{P\left(y_j\right|{\mathrm{\&omega;}}_F)\&times;P\left(y_j\right|{\mathrm{\&omega;}}_N)}\right]$ Calculate predicate p _jsuccess and failure performs the overlapping degree of Spectral structure.Wherein, D _jrepresent predicate p _jthe set that all evaluation error values are formed, y _jrepresent a value in this set, ω _nand ω _fthe program that represents respectively is run successfully and is run unsuccessfully;

(2) predicate p is calculated _jperform the standardization between class distance of Spectral structure, use A _jrepresent.For predicate p _jthe execution spectrum set that assessment deviate is formed respectively when program runs success and failure, first, utilizes formula calculate predicate p _jsuccess and failure performs the distance between Spectral structure, i.e. between class distance.Wherein, predicate p when representation program runs successfully _jthe average of all evaluation error values. predicate p when representation program runs unsuccessfully _jthe average of all evaluation error values.Then, formula is utilized $D_j=\frac{\sqrt{\frac{{\mathrm{\&Sigma;}}_{i=1}^v\left[{(x(p_j,f_i)-m_j^F)}^2\right]}{v}}+\sqrt{\frac{{\mathrm{\&Sigma;}}_{i=1}^u\left[{(x(p_j,n_i)-m_j^N)}^2\right]}{u}}}{2}$ Calculate predicate p _jsuccess and failure performs the degree of Spectral structure inner dispersion, i.e. inter-object distance.Finally, formula is utilized ask for

Predicate p _jperform the standardization between class distance of Spectral structure.If it should be noted that D _j=0, and B _j≠ 0, then A is set _j=+∞.If D _j=B _j=0, then A is set _j=0;

(3) predicate p is calculated _jdefect Correlation degree.By the result obtained respectively in two steps above, utilize formula calculate predicate p _jdefect Correlation degree.

(4) above-mentioned to each pitching pile predicated execution in program (one) to (three) step, until obtain the Defect Correlation degree of whole predicate.And according to Defect Correlation degree from high to low, all predicates are sorted.It should be noted that if run into the identical situation of the suspicious degree of predicate, then according to the ascending sequence of predicate numbering.In addition, for the O that may occur in previous step _j=A _jthe special circumstances of=0, need this predicate to come after all predicates in sorting operation, because the success and failure execution spectrum of this predicate is identical, show as least suspicious;

5th step, according to the predicate sequencing table obtained, is sequentially searched predicate, until find defect associated predicate.

The present invention compared with the conventional method compared with advantage be: the present invention allows the conforming existence of coincidence, and utilizes the correlation properties of coincidence consistency problem to carry out defect location, thus improves the positioning precision of algorithm.And method is simple, easily realizes.

Accompanying drawing explanation

Fig. 1 is example procedure code

Fig. 2 is the partial information schematic diagram that pile function exports

Fig. 3 is schematic flow sheet of the present invention

Fig. 4 is program details figure to be measured

Fig. 5 is experimental result contrast in all programs to be measured

Fig. 6 is experimental result contrast on print_tokens and print_tokens2

Fig. 7 is experimental result contrast on replace

Fig. 8 is experimental result contrast on schedule and schedule2

Fig. 9 is experimental result contrast on tcas

Figure 10 is experimental result contrast on tot_info

Figure 11 is experimental result contrast on space

Figure 12 is experimental result contrast on flex

Figure 13 is experimental result contrast on grep

Symbol mod sum label in figure is described as follows:

Fig. 1 is the segment program code intercepted from one large section of program, and every a line is a code, and foremost one row are code place line numbers, and middle row are program codes, and wherein, defect is positioned at the 5th row.The most right side one row are predicate to this section of program pitching pile and corresponding predicate numbering.

The operation result of all pitching pile predicates in program to be measured when every a line in Fig. 2 represents that test case is run.Wherein, B represent the statement of pitching pile be branch statement or quantity to statement, the first number after B in bracket is the evaluation error value of this pitching pile predicate.R represents that the statement of pitching pile is return statement, and that number maximum in bracket after R is the evaluation error value of this pitching pile predicate.

First row print_tokens, print_tokens2, schedule, schedule2, replace, tot_info, tcas, space, flex and grep in Fig. 4 are respectively the title of program to be measured; #of selected versions is the defect version number selected by each program to be measured, and each defect version only comprises a defect; #of LOC is the code length of each program, and unit is row; #of predicates is the predicate number of pitching pile in each program to be measured; #of runs is the test case number running program to be measured.

Horizontal ordinate in Fig. 5 ~ Figure 13 is the number percent that the predicate number checked accounts for whole predicate, and namely search cost, ordinate is the number percent that the defect navigated to accounts for overall defect.In figure, J-B is that the present invention marks, and CBI, SOBER, Wilcoxon and Mann-Whitney are respectively each algorithm title.

Embodiment

Assuming that containing several defects in program to be measured, before test, these defects are unknown usually.First carry out pitching pile to program to be measured, the output information of design stake point is the information described in Fig. 2.Then load test case, run the program after pitching pile, output state information.Utilize these status informations to calculate the Defect Correlation degree of predicate, and arranged from high to low according to Defect Correlation degree by predicate, the last predicate sequence according to obtaining from up to down finds defect in a program.

In order to check the location efficiency of institute's extracting method in this patent, consider using these routine packages of flex and grep(in Siemens routine package, space routine package and UNIX routine package it is all generally acknowledged typical tested object) as experimental subjects, defective locations, defect counts and defect type in these programs to be measured are all known.Fig. 4 shows the details of all programs to be measured.

Choose section program of shown in Fig. 1 to verify the inventive method.Use flow process of the present invention as shown in Figure 3, its concrete implementation step is as follows:

The first step carries out pitching pile to this section of program.Insert 7 stake points altogether, the position of pile function and numbering as shown in Figure 1, arrange the information format of pile function output as shown in Figure 2;

Second step loads 6 test cases, and they are T1:(2 respectively, and 1,3), T2:(2,1,2), T3:(1,1,3), T4:(1,2,3), T5:(2,3,1), T6:(1,2,0) program after pitching pile, is run.Wherein, front four test cases make program run successfully, and latter two test case makes program run unsuccessfully.There is the conforming test case of coincidence in T3 and T4;

3rd step collects the predicate Dynamic Execution information that when every secondary program is run, pile function exports respectively.Such as, when program is run successfully, predicate P ₄obtain 4 evaluation error values altogether, be respectively 0.5,0.5,0,0.When program is run unsuccessfully, predicate P ₄obtain 2 evaluation error values altogether, be respectively 0,0;

4th step carries out statistical analysis to all predicate Dynamic Execution information collected, and this step can be divided into four small steps again:

(1) predicate P is calculated ₄success and failure performs the overlapping degree of spectrum, uses O ₄represent.Through statistics, p (0.5| ω _n)=0.5, p (0.5| ω _f)=0, p (0| ω _n)=0.5, p (0| ω _f)=1.Bring formula into $-\ln \left[{\mathrm{\&Sigma;}}_{y_j\&Element;D_j}\sqrt{P\left(y_j\right|{\mathrm{\&omega;}}_F)\&times;P\left(y_j\right|{\mathrm{\&omega;}}_N)}\right],$ O can be obtained ₄=0.347;

(2) predicate P is calculated ₄perform the standardization between class distance of Spectral structure, use A ₄represent.As calculated, b ₄=0.25.Meanwhile, D ₄=0.125.Bring formula into a can be obtained ₄=2;

(3) by O that above-mentioned two steps calculate ₄and A ₄result brings formula into obtain P ₄defect Correlation degree S ₄=0.318;

(4) for each predicate in this section of program, repeat (one) to (three) step, obtain the Defect Correlation degree of each predicate in program.

(5) sorted from high to low according to Defect Correlation degree by all 7 predicates, the sequence of gained predicate is P ₂, P ₃, P ₄, P ₅, P ₆, P ₇, P ₁.

5th step utilizes this sequence to search successively predicate.Known predicate P ₄be defect predicate, then in this sequence, find the 3rd predicate and can navigate to defect.

Said method is used to carry out defect location to whole 171 programs to be measured, and choose CBI algorithm, SOBER algorithm, Wilcoxon algorithm and Mann-Whitney algorithm as comparison algorithm (choosing CBI algorithm and SOBER algorithm is because these two kinds of algorithms are typical algorithm, choose Wilcoxon algorithm and Mann-Whitney algorithm is because these two kinds of algorithms are proved to be at present and can obtain good locating effect).In addition, the evaluation criterion of P-score as algorithm locating effect is introduced.Wherein, the total number of pitching pile predicate in L representation program, represent defect associated predicate, represent the ranking of defect associated predicate in predicate sorted lists (such as, during upper mask body is implemented for example, ), P-score less expression location efficiency is higher.

As can see from Figure 5, for whole 171 programs to be measured, the inventive method (with J-B mark in figure, following figure is similar) all shows at all statistics Nodes of transverse axis the locating effect being better than other algorithms.Such as, when the predicate of inspection 10%, the inventive method can navigate to defect (47.95% × 171 ≈ 82 of 47.95%, half close to whole defect sum), by contrast, CBI algorithm, SOBER algorithm, Wilcoxon algorithm and Mann-Whitney algorithm navigate to the defect of 21.64%, 12.28%, 32.75% and 18.71% respectively, and visible the present invention obviously will be better than CBI algorithm, SOBER algorithm, Wilcoxon algorithm and Mann-Whitney algorithm in overall locating effect.

Fig. 6 ~ Figure 13 each single program is applied respectively the experimental result that the inventive method and CBI algorithm, SOBER algorithm, Wilcoxon algorithm and Mann-Whitney algorithm carry out defect location to contrast.Can find from Fig. 6 ~ Figure 13, except on some statistics node of Fig. 8 and Figure 11, locating effect of the present invention need except raising, for each single program, the inventive method still can show optimum locating effect.

Draw the following conclusions from above experimental data:

(1) the present invention proposes to utilize the conforming correlation properties of coincidence to carry out defect location, under the prerequisite allowing coincidence consistance to exist, and the impact that the method can effectively avoid coincidence consistance to cause algorithm positioning precision;

(2) location efficiency of the present invention when locating most of bugs is all higher than CBI algorithm, SOBER algorithm, Wilcoxon algorithm and Mann-Whitney algorithm.

Claims (1)

1. towards a statistics software defect positioning method for coincidence consistency problem, it is characterized in that: the method concrete steps are as follows:

Step one: carry out pitching pile to containing defective program; Pitching pile is to obtain the multidate information of specific program node when program is run, here to statement, pitching pile is carried out to the branch statement in program, return statement and quantity, pitching pile is achieved by insertion Boolean expression and predicate, pile function requires the Dynamic Execution information of the outer predicate when program is run, and the Dynamic Execution information of predicate refers to the evaluation error of predicate here; The definition of evaluation error is: in a secondary program operation, predicate is judged as the probability of "True", uses expression formula represent, wherein, e _t(p _j, r _i) and e _f(p _j, r _i) distinguish predicate p when representation program runs for i-th time _jbe judged as "True" and the number of times being judged as "false"; If it should be noted that, predicate is not performed in certain secondary program is run, then evaluation error value is now set to 0.5; For ease of hereinafter illustrating, use p ₁, p ₂..., p _mm pitching pile predicate in representation program;

Step 2: whole test cases of loading procedure, runs the program being inserted with pile function; To be exported by the program of contrast prototype version namely not containing any defect and namely defect version is implanted the program after individual defect and exported, distinguish make program runs successfully with the failed test case of operation; Wherein, use u representation program to run successful number of times, v representation program runs failed number of times;

Step 3: collect the predicate Dynamic Execution information that when every secondary program is run, pile function exports; When program is run successfully, predicate p _jobtain u evaluation error altogether, when program is run unsuccessfully, predicate p _jobtain v evaluation error altogether, wherein, j ∈ [1, m];

Step 4: carry out statistical analysis to all predicate Dynamic Execution information collected, this step is divided into again four small steps:

(1) predicate p is calculated _jsuccess and failure performs the overlapping degree of Spectral structure, uses O _jrepresent; For predicate p _jthe execution spectrum set that assessment deviate is formed respectively when program runs success and failure, utilizes probability distribution criterion $O_j=-\ln \left[{\mathrm{\&Sigma;}}_{y_j\&Element;Y_j}\sqrt{P\left(y_j\right|{\mathrm{\&omega;}}_F)\&times;P\left(y_j\right|{\mathrm{\&omega;}}_N)}\right]$ Calculate predicate p _jsuccess and failure performs the overlapping degree of Spectral structure; Wherein, Y _jrepresent predicate p _jthe set that all evaluation error values are formed, y _jrepresent a value in this set, ω _nand ω _fthe program that represents respectively is run successfully and is run unsuccessfully; Wherein, j ∈ [1, m];

(2) predicate p is calculated _jperform the standardization between class distance of Spectral structure, use A _jrepresent; For predicate p _jthe execution spectrum set that assessment deviate is formed respectively when program runs success and failure, first, utilizes formula calculate predicate p _jsuccess and failure performs the distance between Spectral structure, i.e. between class distance; Wherein, predicate p when representation program runs successfully _jthe average of all evaluation error values; predicate p when representation program runs unsuccessfully _jthe average of all evaluation error values; Then, formula is utilized $D_j=\frac{\sqrt{\frac{{\mathrm{\&Sigma;}}_{i=1}^v\left[{(x(p_j,f_i)-m_j^F)}^2\right]}{v}}+\sqrt{\frac{{\mathrm{\&Sigma;}}_{i=1}^u\left[{(x(p_j,n_i)-m_j^N)}^2\right]}{u}}}{2}$ Calculate predicate p _jsuccess and failure performs the degree of Spectral structure inner dispersion, i.e. inter-object distance; Finally, formula is utilized ask for predicate p _jperform the standardization between class distance of Spectral structure; If it should be noted that D _j=0, and B _j≠ 0, then A is set _j=+∞; If D _j=B _j=0, then A is set _j=0;

(3) predicate p is calculated _jdefect Correlation degree; By the result obtained respectively in two steps above, utilize formula calculate predicate p _jdefect Correlation degree;

(4) above-mentioned to each pitching pile predicated execution in program (one) to (three) step, until obtain the Defect Correlation degree of whole predicate; And according to Defect Correlation degree from high to low, all predicates are sorted; It should be noted that if run into the identical situation of the suspicious degree of predicate, then according to the ascending sequence of predicate numbering; In addition, for the O that may occur in previous step _j=A _jthe special circumstances of=0, need this predicate to come after all predicates in sorting operation, because the success and failure execution spectrum of this predicate is identical, show as least suspicious;

Step 5: according to the predicate sequencing table obtained, sequentially search predicate, until find defect associated predicate.