CN112699053A - Software testing method integrated with fuzzy clustering - Google Patents

Abstract

The invention discloses a software testing method integrated with fuzzy clustering, aiming at applying the fuzzy clustering method to software testing, clustering variants simulating real defects based on similarity, and only detecting similar defects once to improve the efficiency of software defect detection; firstly, generating variants based on a weak variation test criterion, and calculating the similarity between the variants by adopting a mathematical statistics method; then constructing a similarity matrix, finally, sequencing the variants based on the number of the variants similar to the variants, and then fuzzy clustering the variants based on the similarity among the variants, so that the non-central variants are distributed into a plurality of clusters, which is favorable for improving the efficiency of generating test data; the fuzzy clustering method is beneficial to reducing the cost of variation test; the method has great potential for improving the effectiveness and the practicability of the variation test.

Description

Software testing method integrated with fuzzy clustering

Technical Field

The invention relates to the field of computer software testing, and designs a software testing method integrated with fuzzy clustering. The method is different from the original method in that the fuzzy clustering method is applied to software testing, the variants simulating real defects are clustered based on similarity, similar defects only need to be detected once, and the efficiency of software defect detection is improved.

Background

Software testing is an effective way to improve the quality of software products. The software quality is more and more paid attention and concerned by people, the software test is an important means for ensuring the software quality, and through the test, not only can the possible defects of the software be detected, but also the reliability of the software can be improved. The variation test has the obvious advantages of strong debugging capability, convenience, flexibility, high automation degree and the like, and is widely used for evaluating the sufficiency of a test data set and the effectiveness of a software test technology

The mutation test is to insert the real defect of the simulation software into the original program, and the program inserted with the defect is called as a variant; the statement after the change is called a variant statement; and respectively executing the original program and the variant by using the same test input, if certain test data can distinguish the original program from the variant from an execution result, the variant is called to be killed based on a strong variation test criterion, and if the original program and the variant are executed at a variation point and the variable state is inconsistent, the variant is called to be killed based on a weak variation test criterion. The mutation score is the ratio of killed variants to non-equivalent variants and is used for evaluating the defect detection capability of the test data set, and the higher the mutation score is, the stronger the actual defect detection capability of the test data set is.

A program often has many variants, and in order to kill these variants, a large amount of test data is also required; moreover, in order to generate an effective test data set and kill variants to the maximum extent, the original program and variants must be repeatedly executed, which increases the cost of the variant test and seriously hinders the wide application thereof in software engineering, and therefore, a suitable method needs to be researched to improve the efficiency of the variant test.

Studies have shown that reducing the number of runs of variants is one of the effective ways to improve the efficiency of the variation test. Offutt et al selects representative operators for all operators to generate variants to reduce the number of variants that need to be killed. Mateo et al propose to reduce the number of variants by constructing additional code frames, studying the factors that affect the coverage of variants. In domestic research, the variants generated by similar mutation operators are reduced by the Chengzhou universities team, so that the variants needing to be killed are reduced. The Gongyun team judges the redundancy relation reduction variants among the variants through interval operation, and the consolidation team selects important variant test objects from the aspect of statement importance, reduces the number of variants generated, and groups the variants based on the similarity of the variants according to paths to perform efficient software test.

The clustering method is to classify the data based on data similarity under the unsupervised condition, wherein fuzzy clustering can effectively cluster data sample sets with cross between classes, and the obtained clustering result is obviously superior to a hard clustering method. The fuzzy clustering method is applied to the variation test, variants with high similarity can be classified into the same cluster, the variants are effectively reduced, and the execution cost of the variation test is reduced.

The domestic volitary team is dedicated to research on Software defects predicted based on machine learning, wherein research results based on a clustering method mainly include that a characteristic subset selection framework based on clustering analysis is published in a Computer Software and Applications conference in 2014, and a two-stage data set preprocessing method based on clustering defect prediction is published in a Software Security and Reliability conference; characteristic selection method based on cluster analysis in 'Chinese science' published software defect prediction in 2016, and research on static software defect prediction method in 'software science' published.

Hussain firstly adopts a clustering method to realize the reduction of the variant, and the clustering method used by Hussain comprises a K-means clustering algorithm and an agglomeration type hierarchical clustering algorithm. The method has the disadvantages that the number of clusters and the initial clustering center are difficult to determine in advance by adopting a K-means clustering algorithm, and the clusters cannot be corrected once being combined by adopting an agglomeration type hierarchical clustering algorithm, so that the clustering quality of the variants is influenced. Aiming at the problems, the topaz culvert provides clustering based on genetic algorithm improvement of variants, and the method can reduce the cost of variation test and greatly improve the reduction rate. However, in the above two methods, based on the strong mutation test criterion, the original program and the variants need to be executed by the test data, the calculation cost of the mutation test is still large, and the distance between the test data is not accurate enough as the similarity characteristic of the variants, and for some programs, the test data with a close distance may not necessarily kill the similar variants.

In view of the above analysis, the invention applies the fuzzy clustering and weak mutation testing technology to software testing, can enhance the software testing defect detection capability, reduce the cost of software testing, and is beneficial to the wide application of the mutation testing in the industry.

Disclosure of Invention

The invention provides a software testing method integrated with fuzzy clustering. Firstly, generating variants based on a weak variation test criterion, and calculating the similarity between the variants by adopting a mathematical statistics method; then, constructing a variant similarity matrix; finally, the variants are ranked based on the number of variants that are similar to the variants, and the clustered variants are blurred based on the similarity between the variants, such that non-central variants are assigned to the plurality of clusters.

The technical scheme adopted by the invention is as follows: a software testing method for integrating fuzzy clustering comprises the following steps:

step S1, calculating the similarity between variants

Setting a certain test program as G, inputting the program as X, and s as a certain original sentence in G, and performing mutation on the program to obtain a mutation sentence s'; satisfy the requirement "s!for mutation testing! I.e. meets weak mutation test criteria, where "! If not, then the conditional statement "if s! S' ″ and its true branch, called the variant branch based on weak variant test criteria, and its corresponding variant denoted as M_i(ii) a One variant corresponds to one variant branch; following the same procedure, a set of all variants was obtained, denoted as M ═ M₁,M₂,…,M_nN is the number of variants; inserting the variant branches into the front of the corresponding original sentence in the G to form a new tested program which is marked as G'; x runs G', if "if is! The true branch of s' ″ is executed, then based on the weak mutation criterion M_iIs killed;

to reflect that X killed variant M_iCase, a random variable μ is defined_i(X)，

To calculate μ_i(X) a mathematical statistical method is adopted; first, a test case set is obtained, for which, R samples, denoted X, are randomly generated in the program G input domain₁,X₂,...,X_R(ii) a Mixing X_kK 1,2, …, R performs a procedure to see if it kills M under weak mutation test criteria_iThen calculating mu_i(X_k) A value of (d);

to cluster variants, the similarity between variants needs to be determined. Previous studies found that test data that killed a certain variant may also kill other variants, indicating that there is some association between variants and therefore, similarity between variants can be defined based on this association.

Hypothetical variant M_i,M_jI, j 1,2, n, i ≠ j is two distinct variants, defining two random variables μ_i(X_k) And mu_j(X_k) They respectively reflect M_i,M_jThe likelihood of being killed; then mu_j(X_k) The probability of 1 is expressed as:

M_iand M_jThe similarity between them is marked as alpha_i,jAnd can be represented by the following formula:

from the above formula, α_i,j∈[0,1]。

Step S2 construction of variant similarity matrix

For all variants M₁,M₂,…,M_nSimilarity between variants can be established, and a similarity matrix lambda is recorded as:

M₁ M₂ M_j M_n

step S3 fuzzy clustering variants

Let i cluster be C_iAt the beginning of

Let the threshold be recorded as T e (0,1), and M_iNumber of similar variants eta_iThe initial value is 0;

S31：η_i＝0,i＝1,2,...,n；

s32: investigation of M in Lambda_iN, M, i 1,2_iAnd each M_j,M_j∈M,j＝1,2,...,n,M_j≠M_iAlpha of (A)_i,jValue if α_i,j≥T，η_i＝η_i+1；

S33: based on eta₁,η₂,…,η_nDescending order of M₁,M₂,…,M_n(ii) a Outputting the sequenced variant sequence, and recording as M'₁,M'₂,…,M'_nAnd the ordered set is denoted as S ═ M'₁,M'₂,…,M'_n}；

S34: the variable i is 1;

s35: selecting a head element M 'from S'₁M 'as clustering center'₁Is shown as

Update the cluster to

S36: will be provided with

Deleted from S and M, respectively;

s37: examine in

The corresponding row of the image data is displayed,

and M_j,M_j∈M,j＝1,2,...,n,

Alpha of (A)_i,jValue if α_i,jNot less than T, M may be_jPlacing in C_iThereby obtaining the

Centered cluster

Wherein | c_iL is the number of elements in the cluster,

is the kth element in the ith cluster;

s38: will be provided with

Deleting from S;

S39：i＝i+1；

s310: judging whether variants exist in S, and if so, switching to S34;

s311: export variant clusters, denoted C₁,C₂，…C_mWherein

M is the number of clusters for the cluster center of each cluster.

The invention has the beneficial effects that:

(1) the invention applies the fuzzy clustering method to the variation test to carry out variant clustering, and can improve the efficiency of software test defect detection. When a variation test is carried out, a plurality of variants are often used, one variant represents a defect, and similar variants can be grouped into the same class in order to improve the capability of detecting the defect, so that the variants in the same cluster can be killed at the same time.

(2) The invention provides a method for calculating the similarity of variants. One variant is a program, and the similarity between programs is difficult to directly measure. Therefore, the similarity between variants can be measured by executing their test data, i.e. the similarity of variants is determined by taking the probability that the test data killing one variant kills other variants as the mathematical statistics.

(3) The invention provides a cluster center which is the most similar variant to other variants and is beneficial to killing variants in the same cluster at the same time.

Drawings

FIG. 1 is a general flowchart of a software testing method incorporating fuzzy clustering according to the present invention;

FIG. 2 is an example program in an embodiment of the invention;

FIG. 3 is a similarity matrix between variants;

Detailed Description

For further explanation of the details and advantages of the present invention, reference is now made to the accompanying drawings.

As shown in fig. 1, it is a general flowchart of a software testing method incorporating fuzzy clustering proposed by the present invention. The method comprises the following steps:

step S1, calculating the similarity between variants

Setting a certain test program as G, inputting the program as X, and s as a certain original sentence in G, and performing mutation on the program to obtain a mutation sentence s'; satisfy the requirement "s!for mutation testing! I.e. meets weak mutation test criteria, where "! If not, then the conditional statement "if s! S' ″ and its true branch, called the variant branch based on weak variant test criteria, and its corresponding variant denoted as M_i(ii) a One variant corresponds to one variant branch; following the same procedure, a set of all variants was obtained, denoted as M ═ M₁,M₂,…,M_nN is the number of variants; inserting the variant branches into the front of the corresponding original sentence in the G to form a new tested program which is marked as G'; x runs G', if "if is! The true branch of s' ″ is executed, then based on the weak mutation criterion M_iIs killed;

to reflect that X killed variant M_iCase, a random variable μ is defined_i(X)，

To calculate μ_i(X) a mathematical statistical method is adopted; first, a test case set is obtained, for which, R samples, denoted X, are randomly generated in the program G input domain₁,X₂,...,X_R(ii) a Mixing X_kK 1,2, …, R performs a procedure to see if it kills M under weak mutation test criteria_iThen calculating mu_i(X_k) A value of (d);

to cluster variants, the similarity between variants needs to be determined. Previous studies found that test data that killed a certain variant may also kill other variants, indicating that there is some association between variants and therefore, similarity between variants can be defined based on this association.

Hypothetical variant M_i,M_jI, j 1,2, n, i ≠ j is two distinct variants, defining two random variables μ_i(X_k) And mu_j(X_k) They respectively reflect M_i,M_jThe likelihood of being killed; then mu_j(X_k) The probability of 1 is expressed as:

M_iand M_jThe similarity between them is marked as alpha_i,jAnd can be represented by the following formula:

from the above formula, α_i,j∈[0,1]。

Step S2 construction of variant similarity matrix

For all variants M₁,M₂,…,M_nSimilarity between variants can be established, and a similarity matrix lambda is recorded as:

M₁ M₂ M_j M_n

step S3 fuzzy clustering variants

Let i cluster be C_iAt the beginning of

Let the threshold be recorded as T e (0,1), and M_iNumber of similar variants eta_iThe initial value is 0;

S31：η_i＝0,i＝1,2,...,n；

s32: investigation of M in Lambda_iN, M, i 1,2_iAnd each M_j,M_j∈M,j＝1,2,...,n,M_j≠M_iAlpha of (A)_i,jValue if α_i,j≥T，η_i＝η_i+1；

S33: based on eta₁,η₂,…,η_nDescending order of M₁,M₂,…,M_n(ii) a Outputting the sequenced variant sequence, and recording as M'₁,M'₂,…,M'_nAnd the ordered set is denoted as S ═ M'₁,M'₂,…,M'_n}；

S34: the variable i is 1;

s35: selecting a head element M 'from S'₁As a cluster center, M₁' is represented as

Update clusters to

S36: will be provided with

Deleted from S and M, respectively;

s37: examine in

The corresponding row of the image data is displayed,

and M_j,M_j∈M,j＝1,2,...,n,

Alpha of (A)_i,jValue if α_i,jNot less than T, M may be_jPlacing in C_iThereby obtaining the

Centered cluster

Wherein | c_iL is the number of elements in the cluster,

is the kth element in the ith cluster;

s38: will be provided with

Deleting from S;

S39：i＝i+1；

s310: judging whether variants exist in S, and if so, switching to S34;

s311: export variant clusters, denoted C₁,C₂，…C_mWherein

M is the number of clusters for the cluster center of each cluster.

Analysis of test examples:

the source code of the triangle classifier is shown in FIG. 2 (a). Fig. 2(b) shows a new program, in which 30 variant branches generated based on the weak variant test rule are inserted, and their variant set is M ═ M₁,M₂,...,M₃₀}。

The similarity between variants is calculated according to equation (2). For example, if there are 170 test cases killing M based on weak mutation test criteria₁Then 7 of these test cases can kill M₂And 170 test case kills M₅. Based on the formula (4) can be obtained

And alpha_1,5When 1, obviously, M₁And M₅Has a similarity higher than M₁And M₂The similarity of (c).

Similarly, the similarity between all variants can be obtained and the similarity matrix is constructed as Λ, as shown in fig. 3.

Variants are ordered below. For each variant, the number of high correlations with other variants was counted, based on Λ. Let T equal to 0.5, e.g. M₂₄Examine line 24 α of Λ_i,jMore than or equal to 0.5 element, and statistics shows that 13 variants (including self) and M are present₂₄The correlation is greater than 0.5.

Table 1 ordered variants

Below, cluster variants are blurred. As can be seen from Table 1, the most relevant variant M was found₂₄As C₁Cluster center of, immediately after M₂₄Removed from M and S. Then, according to the similarity matrix Λ, consider M₂₄With other variantsBecause of the similarity of M₁₄And M₃,M₅,M₁₁,M₁₂,M₁₃,M₁₅,M₁₆,M₁₇All the similarity is greater than T ═ 0.5, and they are put into C₁. Then, M is added₃,M₅,M₁₁,M₁₂,M₁₃,M₁₅,M₁₆And M₁₇Is removed from S. Thus, the first cluster is obtained:

C₁＝{M₂₄,M₁,M₅,M₆,M₈,M₁₁,M₁₂,M₁₃,M₂₀,M₂₁,M₂₂,M₂₃,M₂₆}

in S, M₂₁And M₂₆Is the second and third elements because it has been classified as C₁They cannot do clustering centers, so element M is chosen as the head in S₁₅As cluster C₂The cluster center of (2). Repeating the above process until

Finally, 9 variant clusters can be obtained, which are respectively:

C₁＝{M₂₄,M₁,M₅,M₆,M₈,M₁₁,M₁₂,M₁₃,M₂₀,M₂₁,M₂₂,M₂₃,M₂₆}

C₂＝{M₁₅,M₃,M₅,M₁₁,M₁₂,M₁₃,M₂₁,M₂₅,M₂₆,M₂₈,M₂₉,M₃₀}

C₃＝{M₁₉,M₃,M₅,M₁₁,M₁₂,M₁₃,M₂₁,M₂₅,M₂₆,M₂₈,M₂₉,M₃₀}

C₄＝{M₆,M₃,M₅,M₈,M₁₁,M₁₂,M₁₃,M₂₅,M₂₇,M₂₈,M₂₉}

C₅＝{M₂,M₁,M₄,M₅,M₁₁,M₁₂,M₁₃,M₂₅,M₂₇,M₂₈}

C₆＝{M₂₀,M₁,M₅,M₁₁,M₁₂,M₁₃,M₁₈,M₂₀,M₂₁,M₂₂,M₂₃,M₂₆}

C₇＝{M₇,M₃,M₁₀,M₁₁,M₁₂,M₁₃,M₂₅,M₂₇,M₂₈}

C₈＝{M₁₄,M₃,M₅,M₁₁,M₁₂,M₁₃,M₁₅,M₁₆,M₁₇}

C₉＝{M₉,M₃,M₅,M₁₀}

the above examples show that the method of the present invention can accurately group similar variants together, thereby improving the efficiency of software testing.

Claims (3)

1. A software testing method integrated into fuzzy clustering is characterized in that: the method comprises the following steps:

setting a certain test program as G, inputting the program as X, and s as a certain original sentence in G, and performing mutation on the program to obtain a mutation sentence s'; satisfy the requirement "s!for mutation testing! I.e. meets weak mutation test criteria, where "! If not, then the conditional statement "if s! S' ″ and its true branch, called the variant branch based on weak variant test criteria, and its corresponding variant denoted as M_i(ii) a One variant corresponds to one variant branch; following the same procedure, a set of all variants was obtained, denoted as M ═ M₁,M₂,…,M_nN is the number of variants; inserting the variant branches into the front of the corresponding original sentence in the G to form a new tested program which is marked as G'; x runs G', if "if is! The true branch of s' ″ is executed, then, based on the weak varianceDifferent criteria M_iIs killed;

step S1: calculating the similarity between the variants;

step S2: constructing a variant similarity matrix, for all variants M₁,M₂,…,M_nSimilarity between the variants can be established to form a similarity matrix Lambda;

step S3: fuzzy clustering variants:

let i cluster be C_iAt the beginning of

Let the threshold be recorded as T e (0,1), and M_iNumber of similar variants eta_iThe initial value is 0;

S31：η_i＝0,i＝1,2,...,n；

s32: investigation of M in Lambda_iN, M, i 1,2_iAnd each M_j,M_j∈M,j＝1,2,...,n,M_j≠M_iAlpha of (A)_i,jValue if α_i,j≥T，η_i＝η_i+1；

S33: based on eta₁,η₂,…,η_nDescending order of M₁,M₂,…,M_n(ii) a Outputting the sequenced variant sequence, and recording as M'₁,M′₂,…,M′_nAnd the ordered set is denoted as S ═ M'₁,M′₂,…,M′_n}；

S34: the variable i is 1;

s35: selecting a head element M 'from S'₁M 'as clustering center'₁Is shown as

Update clusters to

S36: will be provided with

Deleted from S and M, respectively;

s37: examine in

The corresponding row of the image data is displayed,

and M_j,M_j∈M,j＝1,2,...,n,

Alpha of (A)_i,jValue if α_i,jNot less than T, M may be_jPlacing in C_iThereby obtaining the

Centered cluster

Wherein | c_iL is the number of elements in the cluster,

is the kth element in the ith cluster;

s38: will be provided with

Deleting from S;

S39：i＝i+1；

s310: judging whether variants exist in S, and if so, switching to S34;

s311: export variant clusters, denoted C₁,C₂，…C_mWherein

For each cluster centre, m being clusterAnd (4) the number.

2. The software testing method integrated into fuzzy clustering according to claim 1, wherein: the method for calculating the similarity between the variants in step S1 includes:

defining a random variable mu_i(X) reflects that X kills variant M_iSituation(s)

Calculating mu by adopting a mathematical statistic method_i(X) first, a test case set is obtained, for which R samples, denoted X, are randomly generated in the program G input domain₁,X₂,...,X_R(ii) a Mixing X_kK 1,2, …, R performs a procedure to see if it kills M under weak mutation test criteria_iThen calculating mu_i(X_k) A value of (d);

then, the hypothetical variant M_i,M_jI, j 1,2, n, i ≠ j is two distinct variants, defining two random variables μ_i(X_k) And mu_j(X_k) Respectively reflect M_i,M_jThe likelihood of being killed; then mu_j(X_k) The probability of 1 is expressed as:

M_iand M_jThe similarity between them is marked as alpha_i,jAnd can be represented by the following formula:

from the above formula, α_i,j∈[0,1]。

3. The software testing method integrated into fuzzy clustering according to claim 1, wherein: the method for constructing the variant similarity matrix in the step S2 includes:

for all variants M₁,M₂,…,M_nEstablishing a variant similarity matrix lambda according to the similarity between the variants, and recording the similarity matrix lambda as:

Images