CN108509347B

CN108509347B - Equivalent variant identification method and device

Info

Publication number: CN108509347B
Application number: CN201810359665.XA
Authority: CN
Inventors: 林涛; 刘海陆; 吴芝明; 黎鸣
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2018-04-20
Filing date: 2018-04-20
Publication date: 2020-10-02
Anticipated expiration: 2038-04-20
Also published as: CN108509347A

Abstract

The embodiment of the invention provides an equivalent variant identification method. The equivalent variant identification method comprises the following steps: acquiring a source code of a variant to be identified; generating a graph model of the variant to be identified according to the source code; searching a plurality of propagation paths of the variant to be identified according to the graph model; and obtaining an identification result by searching whether the plurality of propagation paths output the error state generated by the variant to be identified exist.

Description

Equivalent variant identification method and device

Technical Field

The invention relates to the technical field of computers, in particular to an equivalent variant identification method and device.

Background

The mutation testing (mutation testing) is an error-based software testing method, a large number of error programs called variants are generated by injecting various small syntax errors into a program to be tested so as to simulate real errors occurring in the software development process, the effectiveness of a test set is measured according to the capability of the test set for distinguishing an original program from the error programs, if the output of one variant on the test set is different from that of the original program, the variant is called to be killed by the test set, otherwise the variant is called to be alive. Equivalent variants refer to those variants that are semantically (functionally) identical to the original program, although they are syntactically (textually) different from the original program, and because they are functionally identical to the original program, the output on any test set is identical to the original program, i.e., the equivalent variants are functionally completely correct and cannot simulate any errors in the software development process. The problem of equivalent variants has become one of the important obstacles restricting the development of variant tests.

Disclosure of Invention

In view of the above, the present invention provides an equivalent variant identification method and apparatus.

The embodiment of the invention provides an equivalent variant identification method, which comprises the following steps:

acquiring a source code of a variant to be identified;

generating a graph model of the variant to be identified according to the source code;

searching a plurality of propagation paths of the variant to be identified according to the graph model;

and obtaining an identification result by searching whether the plurality of propagation paths output the error state generated by the variant to be identified exist.

The embodiment of the present invention further provides an equivalent variant identification apparatus, which is characterized by including:

acquiring a source code of a variant to be identified;

Compared with the prior art, the equivalent variant identification method and the equivalent variant identification device in the embodiment of the invention have the advantages that the graph model of the variant to be identified is obtained, and then the propagation path of the variant is searched, so that whether a path exists or not can be further judged, the propagation path of an error state generated by the variant to be identified can be output, the identification of the equivalent variant is realized, and the test effect can be further improved when an application program is tested conveniently.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1 is a block diagram of an electronic terminal according to an embodiment of the present invention.

Fig. 2 is a flowchart of an equivalent variant identification method according to an embodiment of the present invention.

Fig. 3 is a flowchart of step S104 of the equivalent variant identification method according to the embodiment of the present invention.

FIG. 4a shows a program dependence graph of variants in one example.

FIG. 4b shows a program dependence graph of variants in one example.

FIG. 5 shows a statistical representation of the identification of variants by the equivalent variant identification method in one example.

Fig. 6 is a functional block diagram of an equivalent variant identification apparatus according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

As shown in fig. 1, is a block diagram of an electronic terminal 100. The electronic terminal 100 includes an equivalent variant identification device 110, a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input/output unit 115, and a display unit 116. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic terminal 100. For example, the electronic terminal 100 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

The memory 111, the memory controller 112, the processor 113, the peripheral interface 114, the input/output unit 115 and the display unit 116 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The equivalent variant identification means 110 includes at least one software functional module which can be stored in the memory 111 in the form of software or Firmware (Firmware) or solidified in an Operating System (OS) of the electronic terminal 100. The processor 113 is configured to execute an executable module stored in the memory, such as a software functional module or a computer program included in the equivalent variant identifying apparatus 110.

The Memory 111 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 111 is configured to store a program, and the processor 113 executes the program after receiving an execution instruction, and the method executed by the electronic terminal 100 defined by the process disclosed in any embodiment of the present invention may be applied to the processor 113, or implemented by the processor 113.

In this embodiment, an Android operating system (Android) is installed in the electronic terminal 100. The Android operating system is stored in the memory 111. Wherein the operating system may include various software components and/or drivers for managing system tasks such as memory management, storage device control, power management, etc.

The processor 113 may be an integrated circuit chip having signal processing capabilities. The Processor 113 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The peripheral interface 114 couples various input/output devices to the processor 113 and memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 may be implemented in a single chip. In other examples, they may be implemented separately from the individual chips.

The input/output unit 115 is used to provide input data to a user. The input/output unit 115 may be, but is not limited to, a mouse, a keyboard, and the like.

The display unit 116 provides an interactive interface (e.g., a user operation interface) between the electronic terminal 100 and a user or is used to display image data to a user reference. In this embodiment, the display unit may be a liquid crystal display or a touch display. In the case of a touch display, the display can be a capacitive touch screen or a resistive touch screen, which supports single-point and multi-point touch operations. The support of single-point and multi-point touch operations means that the touch display can sense touch operations simultaneously generated from one or more positions on the touch display, and the sensed touch operations are sent to the processor for calculation and processing.

The mutation testing (mutation testing) is an error-based software testing method, a large number of error programs called variants are generated by injecting various small syntax errors into a program to be tested so as to simulate real errors occurring in the software development process, the effectiveness of a test set is measured according to the capability of the test set for distinguishing an original program from the error programs, if the output of one variant on the test set is different from that of the original program, the variant is called to be killed by the test set, otherwise the variant is called to be alive. The variation score (mutation score) is used as an index for measuring the effectiveness of the test set, and the calculation formula is as follows:

Ms＝Mk/(Mt-Me)

where, Mk represents the number of variants killed by the test set, Mt represents the number of all variants generated, Me represents the number of equivalent variants, and the equivalent variants refer to those variants that are semantically (i.e., at a functional level) identical to the original program although they are syntactically (i.e., at a textual level), and because the equivalent variants are functionally identical to the original program, the output on any test set is identical to the original program, i.e., the equivalent variants are functionally completely correct and cannot simulate any errors in the software development process.

For example, during a certain test run, 1100 variants are formed, with 800 variants on the test set outputting results different from the original program (killed), 300 variants outputting results identical to the original program (surviving), and there are two possible cases for 300 surviving variants: the first is that the test set is insufficient, and a test case capable of killing a survival variant is lacked; and the second is that the survival variant is an equivalent variant and cannot be killed by any test case. Equivalent variants therefore need to be identified among the surviving variants and removed from the set of variants before calculating the variation score.

The problem of the equivalent variants becomes one of the important obstacles restricting the development of the variation test, firstly, the existence of the equivalent variants can not enable testers to accurately calculate the variation score, and the accuracy of the test result is seriously influenced; secondly, the number of equivalent variants is huge and difficult to identify, and the test cost is greatly improved; thirdly, the existence of equivalent variants seriously hinders the automated running of mutation tests. The patent provides an automatic equivalent variant identification method based on a program dependency graph aiming at the equivalent variant problem in a variant test, and the identification problem of the equivalent variant is well solved.

Thus, the present invention can identify variants through the following examples, which are described in detail below.

Please refer to fig. 2, which is a flowchart illustrating an equivalent variant identification method applied to the electronic terminal 100 shown in fig. 1 according to an embodiment of the present invention. The specific process shown in fig. 2 will be described in detail below.

And step S101, acquiring a source code of the variant to be identified.

And S102, generating a graph model of the variant to be identified according to the source code.

In this embodiment, the step S102 includes: and generating a program dependence graph of the variant to be identified according to the source code.

And S103, searching a plurality of propagation paths of the variant to be identified according to the graph model.

In this embodiment, the step S103 includes: and acquiring all data propagation paths and control propagation paths corresponding to the program dependency graph.

The data propagation path represents a path which takes a variant node as a starting point and takes any output node as an end point in a program dependency graph of the variant, wherein a first edge from the variant node is a data dependent edge related to a propagation variable.

The control propagation path is represented by a path which takes a variant node as a starting point and takes any output node as an end point in a program dependency graph of the variant, wherein a first edge from the variant node is a control dependency edge.

And step S104, finding whether the propagation paths outputting the error state generated by the variant to be identified exist in the plurality of propagation paths to obtain an identification result.

In this embodiment, as shown in fig. 3, the step S104 includes: step S1041 and step S1042.

Step S1041, determining a variation node of the variant to be identified and a propagation variable generated by the variation node according to the source code.

In the present embodiment, the propagation variable represents a variable in which a state change occurs at a mutation node.

In this embodiment, the step S1041 includes: and comparing the variant to be identified with the original program corresponding to the variant to be identified to determine a variant node and a propagation variable generated by the variant node.

Step S1042, determining whether the multiple propagation paths include a change output result derived from outputting the propagation variable, and if no path exists in the multiple propagation paths that outputs the change output result derived from outputting the propagation variable, determining that the to-be-identified variant is an equivalent variant.

In this embodiment, by comparing the abstract syntax trees before and after the variation of the variant sentence of the variant to be identified, which propagation variables exist in the variant can be analyzed, and meanwhile, whether the expression value of the variant sentence can be changed before and after the variation can be determined.

Optionally, when the statement represented by the variant node is a control statement, if the value of the expression of the variant node can be changed, the data propagation path and the control propagation path need to be searched in the program dependency graph of the variant at the same time; otherwise, only the data propagation path needs to be searched.

The above embodiments are described in detail below by way of a few examples.

For variants which cannot be killed by any test case and are called equivalent variants, the identification problem of the variants can be converted into a path search problem in a program dependency graph in the example, namely whether a path exists in the program dependency graph or not is judged, and an error state generated by a variant node of the variant to be identified can be propagated to any output node.

In one example, the original program corresponding to the variant to be identified may be an isTriangle program, specifically:

as shown in FIG. 4a, FIG. 4a shows a program dependence graph of variants in one example. FIG. 4 is a program dependency graph showing a variant of the above example isTriangle program.

In the above example, a < 0 at S2 is mutated into +++ a < 0, the mutation node is a control statement, the propagation variable is a, and the value of the statement expression is changed, so the above variants search for the data propagation path and the control propagation path at the same time. As shown in fig. 4a, the data propagation paths are (S2, S4, S5, S6), the control propagation path is (S2, S3, S6), and the variant has two propagation paths, so that a propagation path capable of outputting an error state generated by a mutation node can be found in the program dependency graph of the variant, and thus the variant is not an equivalent variant due to lack of dependency.

Further, the variant to be identified may also be a variant generated by deleting a certain node. For example, when a variant is generated by using the mutation operator SDL (specification deletion), the mutation operator SDL may delete a certain statement from the program completely, and it is shown in the program dependency graph that a certain node and all its child nodes are deleted. The judgment is based on that in the original program dependency graph, the edge taking the node as the terminal point is deleted in the variant program dependency graph, or the edge taking the node as the terminal point is newly added in the variant dependency graph. After the affected nodes are determined, the propagation paths of the nodes are searched in the variant program dependency graph as the propagation paths of the variant.

The following are variants produced after the deletion of the node S2 in the original program corresponding to the above variants:

FIG. 4b shows the variant obtained by deleting S2 in the isTriangle program by using the SDL mutation operator, and comparing the program dependency graphs before and after mutation, the program dependency graph after mutation lacks a data dependent edge (S3, S6, result), so the set of affected nodes is { S6}, the propagation path of the variant is (S6), so the variant is not the equivalent variant due to lack of dependency relationship.

The embodiment of the invention can effectively avoid two defects of the existing data flow analysis method by searching the propagation path by using the program dependence graph: (1) two main dependency relationships exist in a program, namely a data dependency relationship and a control dependency relationship, the existing data flow analysis method only considers the data dependency relationship in the program but not the control dependency relationship in the program, and the equivalent variants generated due to the lack of the control dependency relationship cannot be identified. (2) The existing data flow analysis method is based on a fixed data flow mode, and once invalid sentences which do not influence the output result of a program exist in the program, the fixed data flow mode can be damaged, and equivalent variants cannot be identified.

In order to further verify the technical effect of the present patent, the applicant conducted a comparative experiment on the method proposed in the embodiment of the present invention and an existing data flow analysis method. The equivalent variant identification method based on the program dependency graph provided by the embodiment of the invention can well solve two main problems of the existing data flow analysis method, can accurately identify the equivalent variants generated due to the lack of dependency relationship between the variant statement and the output statement, can identify 70.49% of the equivalent variants on average in some experimental examples, is improved by 12.08% on average compared with the existing data flow analysis method, and particularly has outstanding identification capability on the equivalent variants generated by the AOIS variant operator.

During the experiment, 2574 variants were co-generated, of which 389 equivalent variants account for 15.11% of the total variants, and the experimental results are shown in table 1, in which the third column reports the number of equivalent variants identified by the dataflow-pattern-based method, and 60.41% of equivalent variants were identified on average in this experiment. The fourth column reports the number of equivalent variants identified using the equivalent variant identification method in the present example, which can identify on average 72.49% of equivalent variants; the fifth column records the improvement of the equivalent variant identification method in the embodiment of the invention relative to a data stream mode-based method, and the equivalent variant identification rate which can be improved by 12.08% on average by using the method provided by the equivalent variant identification method in the embodiment of the invention.

Table 1:

for further analysis on whether the method provided by the equivalent variant identification method in the embodiment of the invention solves the problems existing in the existing data stream analysis method, the equivalent variants are counted according to the types of the mutation operators of the equivalent variants, the statistical result is shown in fig. 5, and for the equivalent variants generated by the AOIS mutation operator and the SDL mutation operator, the method provided by the equivalent variant identification method in the embodiment of the invention is obviously superior to the existing data stream analysis method. On one hand, for the equivalent variants generated by the AOIS mutation operator, the existing data flow analysis method detects whether a propagation variable generated at a mutation statement is referred by other statements before the next assignment based on a fixed data flow mode, if the propagation variable is not referred by any other statement before the next assignment, the variant is judged to be the equivalent variant, so that the method can only identify the equivalent variant of which the propagation variable does not propagate backwards, therefore, when some statements which refer to the propagation variable but do not influence the program output exist in the program, the original data mode is broken, and the identification of the equivalent variant is interfered, while the method provided by the equivalent variant identification method in the embodiment of the invention detects whether the dependency relationship exists between the mutation statement and the output statement based on a program dependency graph, even if the statements which refer to the propagation variable but do not influence the program output exist in the program, the dependency relationship between the output statement and the variant statement cannot be influenced, so that the method provided by the equivalent variant identification method in the embodiment of the invention is not influenced by invalid statements; on the other hand, for the equivalent variant generated by the SDL mutation operator, the existing data flow analysis method does not consider the control dependency existing in the program, and therefore when the mutation operator deletes a certain statement in the original program, it cannot judge whether the deleted statement affects the output of the program, and therefore it cannot identify the equivalent variant.

According to the equivalent variant identification method provided by the embodiment of the invention, the graph model of the variant to be identified is obtained, and then the propagation path of the variant is searched, so that whether a path exists or not can be further judged, and the propagation path of the error state generated by the variant to be identified can be output, and therefore, the variant is identified, and the test effect can be further improved when an application program is tested conveniently.

Please refer to fig. 6, which is a functional block diagram of the equivalent variant identifying apparatus 110 shown in fig. 1 according to an embodiment of the present invention. Each module and unit in the equivalent variant identifying device 110 in this embodiment are used for executing the steps in the above method embodiments. The equivalent variant identifying means 110 includes: an acquisition module 1101, a generation module 1102, a lookup module 1103, and an identification module 1104.

The obtaining module 1101 is configured to obtain a source code of the variant to be identified.

The generating module 1102 is configured to generate a graph model of the variant to be identified according to the source code.

The searching module 1103 is configured to search multiple propagation paths of the variant to be identified according to the graph model.

The identifying module 1104 is configured to obtain an identifying result by searching whether there is a propagation path that outputs an error state generated by the variant to be identified in the plurality of propagation paths.

In this embodiment, the generating module 1102 is further configured to generate a program dependency graph of the variant to be identified according to the source code.

In this embodiment, the searching module 1103 is further configured to obtain all data propagation paths and control propagation paths corresponding to the program dependency graph.

In this embodiment, the identifying module 1104 includes: the device comprises a determining unit and a judging unit.

And the determining unit is used for determining the variant node of the variant to be identified and the propagation variable generated by the variant node according to the source code.

The judging unit is configured to judge whether the plurality of propagation paths include a change output result derived from outputting the propagation variable, and determine that the variant to be identified is an equivalent variant if there is no path from the plurality of propagation paths that outputs the change output result derived from outputting the propagation variable.

In this embodiment, the determining unit is further configured to compare the variant to be identified with an original program corresponding to the variant to be identified, and determine a variant node and a propagation variable generated by the variant node.

Further details regarding the present embodiment may further refer to the description in the above method embodiments.

The equivalent variant identification device of the embodiment of the invention further judges whether a path exists to output the propagation path of the error state generated by the variant to be identified by obtaining the graph model of the variant to be identified and then searching the propagation path of the variant, thereby realizing identification of the variant and further facilitating improvement of the test effect when testing the application program.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An equivalent variant identification method, comprising:

acquiring a source code of a variant to be identified;

determining a variation node of the variant to be identified and a propagation variable generated by the variation node according to the source code;

and judging whether the plurality of propagation paths include a change output result led out by outputting the propagation variable, and if the plurality of propagation paths do not have a path for outputting the change output result led out by the propagation variable, judging that the variant to be identified is an equivalent variant.

2. An equivalent variant calling method as claimed in claim 1, wherein said step of generating a graph model of said variant to be called from said source code comprises:

and generating a program dependence graph of the variant to be identified according to the source code.

3. An equivalent variant identification method as claimed in claim 2, wherein said step of finding a plurality of propagation paths of said variant to be identified based on said graph model comprises:

and acquiring all data propagation paths and control propagation paths corresponding to the program dependency graph.

4. The equivalent variant identification method of claim 1, wherein the step of determining the variant node of the variant to be identified and the propagation variable generated by the variant node according to the source code comprises:

and comparing the variant to be identified with the original program corresponding to the variant to be identified to determine a variant node and a propagation variable generated by the variant node.

5. An equivalent variant identification device, comprising:

the acquisition module is used for acquiring a source code of the variant to be identified;

the generating module is used for generating a graph model of the variant to be identified according to the source code;

the searching module is used for searching a plurality of propagation paths of the variant to be identified according to the graph model;

the identification module is used for obtaining an identification result by searching whether the plurality of propagation paths output the error state generated by the variant to be identified exist or not;

wherein the identification module comprises: a determination unit and a judgment unit;

the determining unit is used for determining a variation node of the variant to be identified and a propagation variable generated by the variation node according to the source code;

6. The equivalent variant recognition apparatus of claim 5, wherein the generating module is further configured to generate a program dependency graph of the variant to be recognized according to the source code.

7. The apparatus of claim 6, wherein the lookup module is further configured to obtain all data propagation paths and control propagation paths corresponding to the program dependency graph.

8. The apparatus of claim 5, wherein the determining unit is further configured to compare the variant to be identified with an original program corresponding to the variant to be identified to determine a variant node and a propagation variable generated by the variant node.