CN113568831A - Self-supervision deep learning type defect positioning method based on metamorphic test - Google Patents
Self-supervision deep learning type defect positioning method based on metamorphic test Download PDFInfo
- Publication number
- CN113568831A CN113568831A CN202110852911.7A CN202110852911A CN113568831A CN 113568831 A CN113568831 A CN 113568831A CN 202110852911 A CN202110852911 A CN 202110852911A CN 113568831 A CN113568831 A CN 113568831A
- Authority
- CN
- China
- Prior art keywords
- test
- coverage
- test case
- mtg
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Abstract
The invention discloses a self-supervision deep learning type defect positioning method based on metamorphic testing, which comprises the steps of firstly constructing metamorphic relations, generating source test cases and subsequent test cases on the basis of a given test case set without test prediction, then collecting coverage information and test output, then carrying out set solving on the source test cases and the subsequent test cases, carrying out metamorphic relation verification on the subsequent test cases, obtaining the test result of total coverage information in each metamorphic relation group, integrating the test results into a coverage matrix and a violation vector, inputting the coverage matrix and the violation vector to a deep neural network model for training, learning the model, finally completing the prediction of code suspicious values by using the model, sequencing codes of a defect program from large to small according to the suspicious values, and completing the defect positioning. The method overcomes the limitation of deep learning type defect positioning, can be applied to the situation of test prediction missing, realizes the defect positioning facing to the label-free data set in a self-supervision mode, and obviously expands the application range.
Description
Technical Field
The invention relates to the field of defect location, in particular to self-supervision deep learning type defect location to a label-free data set.
Background
Software, as a pure abstract knowledge product with a high degree of inherent complexity, is very immune to errors. Although a great deal of manpower and material resources are invested in reality to try to reduce the vulnerability of the software, the BUG inevitably exists in all the scaled software. Debugging thus becomes a critical part of the process throughout software development and maintenance, and does require a significant amount of labor and considerable time. To alleviate this burden, the academia has devised a number of techniques for automatic BUG localization to help developers find BUGs more efficiently. In recent research in this field, Deep-Learning-based Fault Localization (DLFL) has gained much attention due to its excellent Localization effect. DLFL uses the currently popular supervised deep learning model as its kernel, and can effectively recognize and digest patterns hidden in program execution information, and further uses the learned patterns to evaluate the suspicious values of each line of statements, indicating the responsibility of each line of statements for program failure. Therefore, compared with some other leading edge defect positioning methods (such as defect positioning based on spectrum), the DLFL not only can give better effect, but also has stronger generalization capability.
FIG. 1 shows the basic flow of DLFL. Given a target program (i.e., a bug-locating object, a bug-containing program) P containing N lines of code, a program containing M test cases [ t ]1,t2,…,tM]And a test case set of [ o ] that must be prepared before DLFL starts1,o2,…,oM]Wherein o isiStoring P at tiExpected output of the upper run. In the step of batch execution (batch execution), all test cases are input one by one to the target program for execution. Test case tiIs taken to and the test prediction o corresponding to itiMaking comparison so as to obtain execution result yiTo indicate whether the execution of the program is failed. Meanwhile, the run-time information related to which statements (the statements represent a line of code) are executed is collected and summarized by the relevant tools to obtain xiAnd reserving for follow-up investigation. Next, DLFL uses the matching pairs of execution results (i.e., pass or fail) and runtime information (i.e., statement override), (x)i,yi) I ∈ {1,2, …, M }, the former as a label and the latter as a sample, with the aid of depth scienceAnd (5) learning a network to internalize the relation between the sample and the label and train the model. Further, DLFL evaluates each line of statements for suspect values (i.e., how likely the line of code contains BUG) using a trained model, and calculates [ s [ -s ] is (where-is1,s2,…sN]Wherein s isiThe probability that the ith statement contains a defect is weighed. In the last step, the DLFL sorts all sentences according to suspicious values from large to small, and outputs a ranking list [ l ] of all sentences1,l2,…,lN]Wherein l isiPointing to the code line number where the i-th suspect statement is located.
However, as indicated by the dashed box in FIG. 1, the label is a requirement that the DLFL be able to function. In other words, the supervised learning neural network inside the DLFL requires a large scale tagged dataset. This fact makes DLFL applicable only to those cases where test predictions are readily available. Without the tags, the deep learning model cannot learn which samples will cause program failure and which samples will not cause program failure, and thus cannot learn the association between runtime information and program behavior. In fact, the problem of missing test predictions is often present in practical work, such as numerical calculation type software and machine learning algorithms. Often, accurate or precise test languages are not available at all or are expensive, if possible. This pre-requirement for supervised learning then makes such an efficient technique impractical or unfeasible in many scenarios.
Disclosure of Invention
In view of the above problems in the prior art, the present invention aims to: the method solves the applicability problem of deep learning type defect positioning on the label-free data set, and can be applied to the condition that the data set is short of labels.
In order to solve the technical problems, the invention adopts the following technical scheme: the self-supervision deep learning type defect positioning method based on metamorphic testing comprises the following steps:
s101: constructing a metamorphic relation MR of the target program P according to the prior knowledge;
s102: acquisition testAnd (4) case set t, and picking out all the common M' group source test cases S ═ { S } from the test case set t according to the requirements of the MR1,s2,…,sM′In which s isi＝{si1,si2… shows the ith set of source test cases, sijDenotes siAnd using each source test case group siGenerating a corresponding set of subsequent test cases f for the MRi＝{fi1,fi2…, where a corresponding set of source test cases and subsequent test cases are considered to be the smallest test unit, i.e., a metamorphic test set MTG, the test case contained in the ith MTG is si∪fi；
S103: all test cases in each MTG are input to the target program P one by one for execution, and the execution output of each source test case and subsequent test case in each MTG is collected
And statement coverage matrix during execution
Representing the overlay obtained by executing the jth source test case in the ith group of source test cases,
representing a coverage vector obtained by executing a jth test case in the ith group of subsequent test cases; if it will be
Or
For unifying
Represents, then the elements therein
Indicates whether the k-th statement is overwritten during the execution of the test case c, and if so
Take 1 when not covered
0 is taken, where k has a value in the range of [1, N]N is the total number of defective program code statements;
s104: for each MTG, test case set si∪fiAnd corresponding execution output oiUsing MR to check as parameter, judging whether MR is violated, obtaining test result of MTG
Finally obtaining violation vector composed of M' line test results
Where m is { m ═ m1,m2,…,mM′Represents all M' MTGs, MiRepresents the ith MTG;
s105: overlay matrix for each MTG
Converting to obtain the violation vector y(m)Aligned overlay matrix X(m)；
S106: coverage matrix X obtained using S105(m)And the violation vector y obtained in S104(m)Respectively serving as a sample and a label to train the DLFL model to obtain a trained model F;
s107: constructing an identity matrix X of NxN as an auxiliary coverage matrix for the target program P as follows:
each row represents the covering condition of one virtual test case on N statements, and the i virtual test cases only cover the ith statement;
s108: inputting the auxiliary coverage matrix X into a model F to obtain a suspicious value vector
In which the suspect value of each line of the N lines of code of the defective program P is contained
As an improvement, the coverage matrix of each MTG in S105
Converting to obtain the violation vector y(m)Aligned overlay matrix X(m)The following formula is used:
wherein the content of the first and second substances,
indicating whether the k-th statement is covered in the execution of the test case c; will si∪fiThe coverage condition of the ith MTG at the statement can be obtained by performing logical OR operation on the coverage condition of each test case at the kth statement
The coverage of all N statements constitutes the coverage vector under the MTG
Finally, all M 'coverage vectors form an M' xN coverage matrix X(m)。
Compared with the prior art, the invention has at least the following advantages:
the method of the invention, on the one hand, replaces the test case with the metamorphic relation, and uses it to describe the expected behavior of the program, and on the other hand, defines the metamorphic label by checking whether the predefined metamorphic relation is violated. Through the adjustment of the two aspects, the deep learning type defect positioning can normally work on the original label-free data set.
The method of the invention, combined with metamorphic testing, expands the DLFL method from supervised learning to self-supervised learning, so that the tagged data set is no longer a necessary condition for applying DLFL.
The method of the invention, on the one hand, replaces test cases with metamorphic relations, and uses the metamorphic relations to describe the expected behavior of the program, and on the other hand, defines metamorphic labels by checking whether the predefined metamorphic relations are violated. Through the adjustment of the two aspects, the deep learning type defect positioning can normally work on the original label-free data set.
Drawings
FIG. 1 is a workflow of DLFL.
Fig. 2 is a coverage matrix (sample) and a failure vector (label).
FIG. 3 is a deep learning model of DLFL.
Fig. 4 is an auxiliary coverage matrix.
FIG. 5 is a basic flow diagram of the method of the present invention.
Detailed Description
The present invention is described in further detail below.
DLFL is a localization model that learns to identify statements potentially responsible for program failure using an artificial intelligence neural network containing multiple hidden layers. Specifically, the DLFL takes as input information collected from executing a test case set, then learns a complex nonlinear relationship between sentence behaviors and program failures using a neural network, and after training the model, the trained model is used to evaluate suspicious values of each row of sentences containing defects, and obtain an ordered list of all sentences in descending order of the suspicious values. This section will only show the necessary details in this process.
The information collected from the execution of each test case is divided into two parts, runtime information and execution results, more precisely, coverage information (i.e., whether it is covered) and test results (i.e., whether it is failed).
After a complete execution of a program on a certain test case is completed, statements executed at least once are called to be covered by the test case, and the statements not executed are called to be uncovered. The covered states of all statements constitute a binary vector, where taking 1 means the covered state and taking 0 means covered.
The test result indicates whether the program is operating correctly. Typically, the test results are obtained by comparing the program output to a test prediction that matches the test sample. If they are not the same, it is called failure, and if they are the same, it is regarded as pass. For the purpose of establishing a correlation between a defect statement and a failure result, a failure is represented by 1 and a pass is represented by 0 in the present invention.
In summary, a target program containing N line statements and a test case set containing M test cases can generate coverage vectors and test results as shown in FIG. 2, where x isij＝[test caseicoversj-th statement]，yi＝[test case i results in failure]。[·]Called the evirsen bracket, which is essentially a function of a value range of 0,1, taking a 1 if the condition in the square bracket is satisfied and 0 if not). All the coverage vectors constitute a coverage matrix (i.e., training samples), and all the test results constitute failure vectors (i.e., training labels).
Fig. 3 mainly presents the deep learning process in DLFL. As shown in the figure, the present invention will introduce the teaching of three representative DLFL techniques, which are also used in this experiment. These are CNN-FL using convolutional neural networks, RNN-FL using recurrent neural networks, and MLP-FL using multi-layered perceptrons, respectively. These three types of neural networks have the same interface, i.e., they all have an input layer with N nodes and an output layer with only 1 node. During the training process, they all read in the sample inputs line by line from the coverage matrix, updating the network weights so that the loss of difference between the output of the network and the labels in the failure vector is minimized. In this way, complex non-linear relationships between coverage information and test results can be learned. Thus, the DLFL can generate a trained model, i.e., a probability-based classifier. Given an arbitrary coverage vector, it can predict how likely the coverage condition will cause a program failure.
To use the trained model to judge the suspicious value, the DLFL first needs to construct an auxiliary coverage matrix as a test set. Fig. 4 shows the structure of this auxiliary coverage matrix. It consists of the overlay information of N virtual test cases, where each test case overlays a unique row of statements. Sending it to the trained model for prediction, it can calculate the probability of program failure caused by any single line of statements, i.e. how likely each line of statements, once covered, is to cause program failure. This probability assessment is essentially equivalent to the suspect value of a statement containing a defect. Therefore, the prediction of the auxiliary coverage matrix by the model is used as a suspicious value vector to reflect the defect degree of each row of sentences. And finally, sequencing all the sentences according to the suspicious values by the DLFL to generate a sentence list to be debugged, which is sequenced according to the priority.
Metamorphic Testing (MT) is a practical solution to the problem of missing predicted test predictions in the field of software Testing. How to apply the MT determines that the function of the program is working correctly at a particular input in the absence of a test prediction.
First, a program and a test case are given. The goal is to examine the functional implementation of a program to determine if it is at least able to function properly under a given single test case. In metamorphic testing, it is first necessary to identify at least one MR, an attribute of the program, a necessary basis for functional integrity. During the MT process, several mutually associated test cases are required, each of which plays a different role in the MR. Some of them are source test cases, which are provided by the original test case set; the remaining parts are the subsequent (follow-up) test cases, which are generated during the course of the MT. Given that single test case as the source test case, subsequent test cases can then be generated based on the associations between the different roles defined in the MR. If the input and output of these test cases reveal a violation of the MR, then it can be concluded that the program failed, otherwise the program passed. Because the source test case and the subsequent test case always exist and use as a whole, they are collectively called a Metamorphic Testing Group (MTG). Supplementary explanation one point: multiple source test cases may be stored inside one MTG as long as the MR needs it.
The self-supervision deep learning type defect positioning method based on metamorphic testing comprises the following steps:
s101: constructing a metamorphic relation MR of the target program P according to the prior knowledge; the transmutation relation MR described herein is constructed by those skilled in the art based on a priori knowledge for the target program, and can be accomplished without creative effort, and for the sake of understanding, several rules to be followed by those skilled in the art in constructing MR are given here:
the metamorphic relation MR described here is constructed by those skilled in the art according to a priori knowledge for a target program, and often can be specifically analyzed only for specific problems. In general, to be able to construct a practically effective MR, the following principles need to be followed:
a) and (4) correctness. The MR must reflect the correct behavior of the program function, in other words, if the program function is implemented correctly, the properties specified in the MR must be exhibited.
b) And the subsequent test case is easy to generate. The metamorphic test technique is introduced to solve the problem of test prediction loss at low cost, and if the complexity of the process of generating subsequent test cases based on MR is too high, the aim of low cost cannot be achieved even if the problem can be finally solved, and even the problem cannot be solved.
c) MR with more variation elements or complex expression is preferably selected. In general, the more complex the representation of the MR, the more stringent its specifications for program functional behavior, and thus the higher the error detection capability, the more easily the defects of the program are exposed.
d) The preferences may yield MRs that are unique to subsequent test cases. This criterion requires that no random variables be stored in the MR, so that once the source test case is determined, the subsequent test cases can be uniquely determined. The function of the criterion is to simplify the process of subsequent MR verification, which is beneficial to realizing automation.
It is preferable to select an MR having a wide application range. The source test cases accepted by different MRs are different. If a certain MR can only generate subsequent test cases under a few source test case configurations, the data generated by the MR is very little, and therefore, the practical value is very low.
S102: firstly, a test case set t is obtained, wherein each test case is an input acceptable for a target program P, the storage form is a text file, and data with generality is analyzed from the text file when MR related operation is performed [ the process belongs to the prior art, and is not described herein ]. Then, according to the requirement of MR, picking out all the group source test cases S ═ { S ═ of M' from the test case set t1,s2,…,sM′In which s isi＝{si1,si2… shows the ith set of source test cases, sijDenotes siAnd using each source test case group siGenerating a corresponding set of subsequent test cases f for the MRi＝{fi1,fi2…, where a corresponding set of source test cases and subsequent test cases are considered to be the smallest test unit, i.e., a metamorphic test set MTG, the test case contained in the ith MTG is si∪fi。
Here, each source test case set S in S102 is illustratediGenerating a corresponding set of subsequent test cases f for the MRi＝{fi1,fi2… } processThe following were used:
taking max defect procedure for maximum value as an example, let the expression of MR be max (a, b) + max (c, d)>-max (-b, -a) -max (-d, -c) and where a ≠ b, where the terms on the left of the equation constitute the source set of test cases { (a, b), (c, d) }, and the terms on the right of the equation constitute the terms of the subsequent set of test cases { (-b, -a), (-d, -c) }, and the MR requirement for the source test case is a ≠ b. Now a source test case group s meeting the requirements has been selectediTo obtain a corresponding subsequent test case set, s needs to be firstly determined according to the requirements of (2,5), (3,7) }iSubstituting MR expression to obtain max (2,5) + max (3,7)>-max (-b, -a) -max (-d, -c). Then, according to the mapping relationship between each variable and the source test case in the formula, a-2, b-5, c-3, and d-7 can be obtained. Substituting the values of these variables into the equation, max (2,5) + max (3,7)>Max (-5, -2) -max (-7, -3), which results in the subsequent test case set f contained on the right side of the equationi＝{(-5,-2),(-7,-3)}。
S103: all test cases in each MTG are input to the target program P one by one for execution, and the execution output of each source test case and subsequent test case in each MTG is collected
And statement coverage matrix during execution
Representing the overlay obtained by executing the jth source test case in the ith group of source test cases,
representing a coverage vector obtained by executing a jth test case in the ith group of subsequent test cases; if it will be
Or
For unifying
Represents, then the elements therein
Indicates whether the k-th statement is overwritten during the execution of the test case c, and if so
Take 1 when not covered
0 is taken, where k has a value in the range of [1, N]And N is the total number of defective program code statements. [. the]Called the evirsen bracket, which is essentially a function of a value range of 0,1, taking a 1 if the condition in the square bracket is satisfied and 0 if not).
Acquisition execution output o in the present inventioniAnd statement coverage matrix
The method is the prior art, and specifically comprises the following steps: the method for collecting, executing and outputting comprises two steps, wherein the first step is program output redirection, the content output from the program to the stdout is redirected to a file to obtain original output, and the second step is useful data in the original output is sorted out. The method for collecting the statement coverage matrix also comprises two steps, wherein the first step is to collect original coverage information of program operation by using a coverage tool such as Emma, GCOV and JaCoCo, and the second step is to arrange useful data in the original coverage information, and the statement obtains the coverage matrix.
For example, in S102, a test case group Si∪fiAnd corresponding execution output oiThe procedure for the examination using MR as a parameter is as follows: continuing with the above example based on the defective program max, existing si＝{(2,5),(3,7)}，fiUnder two conditions, { (-5, -2), (-7, -3) }, and the following discussion shows the results of MR examinationTwo results.
a)
Will oiSubstituting MR expression max (a, b) + max (c, d)>-max (-b, -a) -max (-d, -c) to yield 5+7>- (-2) - (-3), further simplification gives 12>5 this question indicates that the original expression is oiIs true, it is concluded that the i-th set of MTGs does not violate the MR.
b)
Will oiSubstituting MR expression max (a, b) + max (c, d)>-max (-b, -a) -max (-d, -c) to yield 5+3>- (-5) - (-7), further simplification gives 8>12 this false proposition, which shows the original expression at oiDoes not hold, and it is concluded that the i-th set of MTGs violates the MR.
S104: for each MTG, test case set si∪fiAnd corresponding execution output oiUsing MR to check as parameter, judging whether MR is violated, obtaining test result of MTG
Finally obtaining violation vector composed of M' line test results
Where m is { m ═ m1,m2,…,mM′Represents all M' MTGs, MiRepresents the ith MTG.
S105: overlay matrix for each MTG
Converting to obtain the violation vector y(m)Aligned overlay matrix X(m)，
The following formula is specifically used:
wherein the content of the first and second substances,
indicating whether the k-th statement is overwritten in the execution of the test case c (see definition in S103); will si∪fiThe coverage condition of the ith MTG at the statement can be obtained by performing logical OR operation on the coverage condition of each test case at the kth statement
The coverage of all N statements constitutes the coverage vector under the MTG
Finally, all M 'coverage vectors form an M' xN coverage matrix X(m)。
S106: coverage matrix X obtained using S105(m)And the violation vector y obtained in S104(m)Respectively serving as a sample and a label to train the DLFL model to obtain a trained model F;
s107: for the target program P, the following N × N identity matrix X is constructed as an auxiliary coverage matrix
Wherein each row represents the coverage of one virtual test case on N statements, and i virtual test cases only cover the ith statement. So configured, the matrix can be used to independently evaluate the relative size of each row of statements responsible for violating MR by fully utilizing the nonlinear relationship learned by the model F.
S108: obtaining the auxiliary coverage matrix X input model FVector of suspect values
In which the suspect value of each line of the N lines of code of the defective program P is contained
Example 1: assume that the program under test is sqrt (x), whose function is to output its square root for an arbitrary input x, and further assume that the given test case is x ═ 2. In this simple scenario, the problem of missing test language is evident, since, on the one hand, it is not possible to detect a language missing
The exact value of (a) is difficult to be calculated by hand, and on the other hand, the existing calculators cannot be relied on, and after all, whether the evolution function provided by the calculators is true and correct cannot be confirmed. Therefore, recourse to metamorphic testing is required. It can be appreciated that the numerous mathematical properties of square root include this term, i.e.
It is now used as one MR. In addition to the source test case of x ═ 2, a random number y needs to be generated as a subsequent test case, for example, y ═ 7, and then another subsequent test case is defined, and z ═ x · y ═ 14. Now, it can be determined whether the target program is correctly implemented by comparing the product of sqrt (2) and sqrt (7) with sqrt (14). If the two are not equal, the program fails, otherwise it passes.
Inputting: the defective program P, which is composed of code of 12 lines N, functions to output the maximum value of two real numbers, as shown below. Where line 5 is the actual location of the defect;
test consisting of 2 test casesSet of use cases t ═ t1,t2The use case inputs and expected outputs are shown in the following table.
| Numbering | Input device | |
| t | ||
| 1 | 2 2 | 2 |
| t2 | 3 1 | 3 |
And (3) outputting: suspect value for each line in N lines of code of defective program P
S101: from the domain knowledge associated with this procedure, an MR: the two input numbers are exchanged for position, and the maximum value is unchanged. The functional interface of the original program is expressed as max (x, y) → z, namely, two real numbers x and y are input, and the program max (x, y) is called to obtain the maximum value z of the two. Then, MR can be expressed as max (x, y) ═ max (y, x).
S102: according to the requirement of MR, all the source test cases S ═ S { S } of the set of original test cases t are picked out1,s2In which s is1＝{s11}，s11＝(2,2)，s2＝{s21}，s21(3, 1). And testing the case set s by each sourceiGenerating corresponding for MRSubsequent test case set to obtain f1＝{(2,2)}，f2{ (1,3) }. A group of related source test cases and subsequent test cases are regarded as minimum test units, namely a Metamorphic Test Group (MTG), test cases contained in the 1 st MTG are { (2,2), (2,2) }, and test cases contained in the 2 nd MTG are { (3,1), (1,3) }.
S103: all test cases are input to P one by one for execution, the execution output of each source test case and subsequent test cases in each MTG and the statement coverage matrix in the execution process are collected to obtain
And
the detailed values are as follows.
o1＝{2,2}
o2＝{1,3}
S104: for each MTG, test case set si∪fiAnd corresponding test output oiUsing MR to check as parameter, judging whether MR is violated, obtaining test result of MTG
Finally, the violation vector is obtained
The specific derivation procedure and results are as follows.
S105: for each MTG, according to the existing coverage matrix
Is converted as follows
Finally, the AND violation vector y is obtained(m)Aligned overlay matrix
The detailed values are as follows.
S106: learning the following model using a deep neural network
F:X(m)→y(m)
And obtaining the trained model F.
In this example, the unbiased multiple regression linear model y ═ f (X) ═ X · w is used, and the model weights are learned as follows.
w＝[0 0 0 0 1 0 0 0 0 0 0 0]T
S107: constructing the following auxiliary coverage matrix as the test set
S108: inputting the auxiliary coverage matrix X into the model F to obtain a suspicious value vector
Containing suspect values for each of the N lines of code of the defective program P, i.e.
The specific derivation procedure and results are as follows.
It can be seen that the suspicious value corresponding to the code in line 5 is 1, and the remaining codes are 0, and actually the code in line 5 is the defect, so the defect location is successful.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all of them should be covered in the claims of the present invention.
Claims (2)
1. The self-supervision deep learning type defect positioning method based on metamorphic testing is characterized by comprising the following steps: the method comprises the following steps:
s101: constructing a metamorphic relation MR of the target program P according to the prior knowledge;
s102: obtaining a test case set t, and picking out all M' groups of source test cases S ═ { S ═ from the test case set t according to the MR requirement1,s2,s,sM′In which s isi＝{si1,si2… shows the ith set of source test cases, sijDenotes siAnd using each source test case group siGenerating a corresponding set of subsequent test cases f for the MRi＝{fi1,fi2…, where a set corresponds toThe source test case and the subsequent test cases of (a) are regarded as the minimum test unit, namely a metamorphic test group MTG, and the test case contained in the ith MTG is si∪fi；
S103: all test cases in each MTG are input to the target program P one by one for execution, and the execution output of each source test case and subsequent test case in each MTG is collected
And statement coverage matrix during execution
Representing the overlay obtained by executing the jth source test case in the ith group of source test cases,
representing a coverage vector obtained by executing a jth test case in the ith group of subsequent test cases; if it will be
Or
For unifying
Represents, then the elements therein
Indicates whether the k-th statement is overwritten during the execution of the test case c, and if so
Take 1 when not covered
0 is taken, where k has a value in the range of [1, N]N is the total number of defective program code statements;
s104: for each MTG, test case set si∪fiAnd corresponding execution output oiUsing MR to check as parameter, judging whether MR is violated, obtaining test result of MTG
Finally obtaining violation vector composed of M' line test results
Where m is { m ═ m1,m2,…,mM′Represents all M' MTGs, MiRepresents the ith MTG;
s105: overlay matrix for each MTG
Converting to obtain the violation vector y(m)Aligned overlay matrix X(m)；
S106: coverage matrix X obtained using S105(m)And the violation vector y obtained in S104(m)Respectively serving as a sample and a label to train the DLFL model to obtain a trained model F;
s107: constructing an identity matrix X of NxM as an auxiliary coverage matrix for the target program P as follows:
each row represents the covering condition of one virtual test case on N statements, and the i virtual test cases only cover the ith statement;
s108: inputting the auxiliary coverage matrix X into a model F to obtain a suspicious value vector
In which the suspect value of each line of the N lines of code of the defective program P is contained
2. The self-supervised deep learning type defect localization method based on metamorphic testing of claim 1, wherein: the coverage matrix of each MTG in the S105
Converting to obtain the violation vector y(m)Aligned overlay matrix X(m)The following formula is used:
wherein the content of the first and second substances,
indicating whether the k-th statement is covered in the execution of the test case c; will si∪fiThe coverage condition of the ith MTG at the statement can be obtained by performing logical OR operation on the coverage condition of each test case at the kth statement
The coverage of all N statements constitutes the coverage vector under the MTG
Finally, all ofThe M 'coverage vectors form a M' xN coverage matrix X(m)。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110852911.7A CN113568831B (en) | 2021-07-27 | 2021-07-27 | Self-supervision deep learning type defect positioning method based on metamorphic test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110852911.7A CN113568831B (en) | 2021-07-27 | 2021-07-27 | Self-supervision deep learning type defect positioning method based on metamorphic test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113568831A true CN113568831A (en) | 2021-10-29 |
| CN113568831B CN113568831B (en) | 2023-07-04 |
Family
ID=78168117
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110852911.7A Active CN113568831B (en) | 2021-07-27 | 2021-07-27 | Self-supervision deep learning type defect positioning method based on metamorphic test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113568831B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572462A (en) * | 2014-12-31 | 2015-04-29 | 中国人民解放军理工大学 | Method for generating metamorphic test case based on adaptive random strategy |
| CN104598381A (en) * | 2015-01-26 | 2015-05-06 | 中国人民解放军理工大学 | Positioning method for failure test instance in metamorphic testing |
| CN104750608A (en) * | 2015-03-25 | 2015-07-01 | 南京大学 | Automatic error locating method based on dynamic symbol operation in procedure |
| US20200257618A1 (en) * | 2019-02-05 | 2020-08-13 | Bank Of America Corporation | System for metamorphic relationship based code testing using mutant generators |
| CN112199295A (en) * | 2020-10-29 | 2021-01-08 | 北京信息科技大学 | Deep neural network defect positioning method and system based on frequency spectrum |
-
2021
- 2021-07-27 CN CN202110852911.7A patent/CN113568831B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572462A (en) * | 2014-12-31 | 2015-04-29 | 中国人民解放军理工大学 | Method for generating metamorphic test case based on adaptive random strategy |
| CN104598381A (en) * | 2015-01-26 | 2015-05-06 | 中国人民解放军理工大学 | Positioning method for failure test instance in metamorphic testing |
| CN104750608A (en) * | 2015-03-25 | 2015-07-01 | 南京大学 | Automatic error locating method based on dynamic symbol operation in procedure |
| US20200257618A1 (en) * | 2019-02-05 | 2020-08-13 | Bank Of America Corporation | System for metamorphic relationship based code testing using mutant generators |
| CN112199295A (en) * | 2020-10-29 | 2021-01-08 | 北京信息科技大学 | Deep neural network defect positioning method and system based on frequency spectrum |
Non-Patent Citations (3)
| Title |
|---|
| HELGE SPIEKER 等: ""Adaptive metamorphic testing with contextual bandits"", 《JOURNAL OF SYSTEMS AND SOFTWARE》, vol. 165, pages 1 - 8 * |
| 张卓 等: ""增强上下文的错误定位技术"", 《软件学报》, pages 266 - 281 * |
| 潘娅 等: ""蜕变测试在机器学习领域的应用"", 《西安邮电大学学报》, pages 81 - 89 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113568831B (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11790256B2 (en) | Analyzing test result failures using artificial intelligence models | |
| US7685082B1 (en) | System and method for identifying, prioritizing and encapsulating errors in accounting data | |
| CN110221975B (en) | Method and device for creating interface case automation test script | |
| WO2021115186A1 (en) | Ann-based program test method and test system, and application | |
| Taylor et al. | Verification and validation of neural networks: a sampling of research in progress | |
| Wong et al. | Using an RBF neural network to locate program bugs | |
| CN113761444A (en) | Code scoring-based course recommendation method, course recommendation device and terminal equipment | |
| CN111581086B (en) | Hybrid software error positioning method and system based on RankNet | |
| CN115168868A (en) | Business vulnerability analysis method and server applied to artificial intelligence | |
| Khaleel et al. | A review paper: optimal test cases for regression testing using artificial intelligent techniques. | |
| CN116776744A (en) | Equipment manufacturing control method based on augmented reality and electronic equipment | |
| Valueian et al. | Constructing automated test oracle for low observable software | |
| CN113568831B (en) | Self-supervision deep learning type defect positioning method based on metamorphic test | |
| CN115827353A (en) | Fault diagnosis method and device | |
| CN111209214B (en) | Code test processing method and device, electronic equipment and medium | |
| Huang et al. | Aligning XAI explanations with software developers’ expectations: A case study with code smell prioritization | |
| US20210397544A1 (en) | Crowdsourced Proactive Testing System for Named Entity Recognition Models in IT Support | |
| Nikiforova et al. | Towards a Business Process Model-based Testing of Information Systems Functionality. | |
| Mulla et al. | Role of Machine Learning & Artificial Intelligence Techniques in Software Testing | |
| CN116485597B (en) | Standardized training method based on post capability model | |
| Mohamad | Machine learning empowered software prediction system | |
| CN110865939A (en) | Application program quality monitoring method and device, computer equipment and storage medium | |
| Yi et al. | A Coverage-Guided Fuzzing Framework based on Genetic Algorithm for Neural Networks | |
| Aho et al. | IVVES (Industrial-Grade Verification and Validation of Evolving Systems). | |
| KR20230054026A (en) | System and method for processing procedures in nuclear power plants |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |