CN111737131B - Code error positioning method based on probability alignment - Google Patents

Abstract

The invention discloses a code error positioning method based on probability alignment. The method is characterized in that a LLVM tool is used for optimizing a source code, extracting symbol information of a program and tracking an execution flow, classifying and storing program examples according to the extracted information to construct a code structure and a symbol execution flow, then aligning the code examples in sequence, carrying out probability alignment on the basis, finally slicing according to key examples of two programs and alignment results, carrying out secondary alignment, focusing on the examples which are matched but not matched, and thus realizing error positioning.

Description

Code error positioning method based on probability alignment

Technical Field

The invention belongs to the technical field of computers, in particular to the technical field of software. The invention provides a code error positioning method based on probability alignment, which can effectively solve the problem that the current OJ (Online Judge) system cannot accurately position the error position in student codes, thereby improving teaching effect and student learning efficiency.

Background

The OJ system is a commonly used system for scoring the programming ability and the understanding level of the programming language of students, and plays an important role in online detection of programming contests, push-to-avoid examinations and the like. According to a plurality of groups of test cases, the current OJ system can compare programs submitted by students and give simple feedback results, such as: accepted, wrong Answer, run Error, time Limited Exceed, memory Limited Exceed, compile Error, etc.

In practical use, the existing OJ system has some problems: (1) After a student gets simple feedback such as Wrong Answer, the student cannot quickly and accurately locate and correct the error code, and a great deal of time is usually consumed for checking and debugging the code; (2) For the questions focusing on the capability of the investigation algorithm, the questions are directly given according to the number of the samples, and the capability of the implementation of the student algorithm cannot be truly reflected, namely the scoring standard needs to be more comprehensive and objective.

An effective solution to the above problem is to combine the correct code information and program analysis techniques when scoring student codes. The method is characterized in that the method is used for extracting the execution path of the correct code and the student code by a dynamic analysis method, and the code statement is subjected to symbol value matching by constraint solving by combining the method of symbol analysis on the basis, so that the error position is positioned and the error position is reasonably scored.

However, due to the complexity of the program language, the execution paths of the codes of different versions often have differences, and the error position cannot be accurately positioned only by symbol value matching. It is therefore a primary object of the present invention to develop a probabilistic method that improves the accuracy of code execution path alignment and achieves false localization. Finally, the correct code and the error code are dynamically analyzed in the execution process, so that the error position is accurately positioned, and correct feedback is given.

Disclosure of Invention

The main work of the invention is to provide a code error positioning method based on probability alignment. First, the present invention focuses on program instruction instances and execution information. The focus is on the matching alignment process performed on the program instruction instances of the correct version and the incorrect version. Secondly, the invention needs to analyze error output according to matching alignment results, find out a key instance, and slice and align again, thereby realizing error positioning. For the above tasks, the work and contributions of the present invention are as follows:

1. the code was optimized using LLVM (Low Level Virtual Machine) to obtain execution information. The invention uses LLVM frame system to convert the test code into language independent intermediate code (IR), optimizes the intermediate language running Pass in turn, classifies and outputs the instance in the program, and at the same time obtains the information of the program running.

2. And restoring and storing the structure and execution information of the source program. Reading in an output file obtained after Pass processing, classifying and storing according to the types of code instances, and constructing a structural framework of a program, wherein each instance has an independent and unrepeated identification id; reading the execution information of the program, associating the identification id with the program information, and recording the execution sequence.

3. The instances are probability aligned. Firstly, screening the code instantiation result, then performing sequence alignment to obtain an initial alignment result meeting a set evaluation rule, and then performing probability alignment (namely probability of the alignment result, which is not one-to-one correspondence) to eliminate error alignment caused by less dependent information during initial alignment, thereby obtaining a more accurate alignment result.

4. The error localization is performed depending on the alignment result. Searching and analyzing the different PRINT examples of the two programs in the example sequences of the two programs after probability alignment to obtain key examples of the two programs, slicing the previously aligned example sequences according to the respective key examples to obtain examples related to the key examples in the programs, and aligning the slices of the two programs for the second time to obtain new slices of the two programs, wherein the slices contain example sequences which are matched but not matched. The result of the slicing is an error instance, the source program code corresponding to the error instance is marked with error information, and a symbol expression value is output, so that error positioning is realized.

Drawings

FIG. 1 is a flow chart of a code error positioning method based on probability alignment

Detailed Description

The method specifically comprises the following steps:

1) And respectively carrying out three-layer optimization on the IR codes generated by the LLVM to obtain instruction examples and execution information.

2) And processing the llvm.out obtained by the Pass optimization, classifying and storing different types of examples, and constructing a structural framework of the source program.

3) Reading in program execution information, corresponding to the source program information stored in the step 2) according to the instance identification id in the execution information, thereby obtaining an execution process and outputting the execution process to a symbol.

4) After obtaining the symbol out generated by the correct version and the error version, performing first sequence alignment on the two versions to obtain an initial alignment result.

5) And (3) carrying out probability-based sequence alignment on the basis of the initial alignment result obtained in the step (4), weakening the possibility of matching errors caused by less matching information in initial matching, and improving the accuracy of the alignment result.

6) And searching and analyzing different PRINT examples in the two aligned program example sequences, and obtaining key examples of the two programs according to the analysis result.

7) Based on the aligned instance sequence, the key instance slice obtained in the step 6) is processed to obtain the instance related to the key instance in the program.

8) And (3) performing second alignment on the slice obtained in the step 7) to obtain new slices of two procedures, wherein the new slices comprise aligned but unmatched example sequences, and simultaneously obtaining new alignment results.

9) The slicing result in the step 8) is the error code. And printing the codes of the error program line by line, marking error information on a certain line of codes if the corresponding example is an example in a slice, and printing symbol expression values.

Step 1) sequentially calling three Pass for codes to be tested through LLVM: constantexpr, mystuff1 and mystuff2.

The constantxpr modifies the IR code and preprocesses it for later optimization. Its task is to handle instruction nesting that occurs in IR code, such as: since the getelementptr instruction embeds the remaining instructions when accessing the array, which is detrimental to subsequent processing, it needs to be disassembled from the nest in the constantexpr, alone as a line of instructions.

mystuff1 will traverse the entire IR code and then format the output program instance information to the log file llvm.out, including: the type of instance; instance identification id, name, dbg information; parameters of instructions or functions, etc.

mystuff2 posts the positions of running entities and instructions such as constants, intermediate variables, function parameters, basic blocks and the like in the IR code, and prints an instance identification id and an entity value into a record file exec.out. The static instance in the llvm.out is related with the dynamic execution process in the exec.out through the id, so that the running track of the code is tracked, and the control flow is obtained.

Step 2) reading in program instance information in llvm.out, and storing according to different instance types, wherein the specific classification and the attribute corresponding to each type are as follows:

step 3) reading in program execution information in exec.out, corresponding to the source program information stored in step 1) according to instance id in the execution information, thereby obtaining an execution process, and outputting the execution process to a symbol.out, wherein the specific steps are as follows:

a) And loading information of each line in exec.out to obtain id and value.

b) And accessing program instructions in the program execution information according to the id, carrying out different processing on different types of instructions, and then calling an execution function to execute. Such as StoreInst and Loadinst require first resolving the memory address of the operation, stroeINst also requires resolving the operand value, which is derived from the value of exec. Then operate in the execute function: fetching dbg information of the instruction, thereby corresponding to the number of source code lines; adjusting the iteration times of the instruction; storing the dynamic execution sequence, and changing the memory information for part of the instructions; instructions are added to the execution trace.

Step 4) performing an initial sequence alignment step, specifically as follows:

a) Loading the content in the symbol.out, and matching the slices of the correct version and the error version in pairs, wherein each matching gives a probability score according to the current two iteration times and the execution order; the more the number of iterations matches the number of executions, the higher the allocation.

b) After the initial assignment is finished, the specific steps for integrating the matching condition are as follows: for the probability score p (i, j) of the i-th item of version 1 and the j-th item of version 2, consider three numbers p (i-1, j-1) +p (i, j), p (i, j-1), and p (i-1, j), which in turn represent three cases where i may match j, may match the j former item, and may match the j latter item; taking the largest one from the three values as a matching result of i, and marking the matching condition of (i, j); and solving an optimal matching result by using a dynamic programming algorithm, thereby obtaining initial sequence alignment.

The specific steps of the step 5) of carrying out sequence alignment based on probability are as follows:

a) Traversing all matching items in initial sequence alignment, and endowing probability initial values; namely:

b) The method changes one-to-one matching into one-to-many matching, and the specific flow is as follows:

traversing all the current matching items, and calling the matching items traversed currently as A-B, wherein A is an item of an error version, and B is an item of a correct version; if the dependency distance of the two items A, B is less than the set threshold (the dependency distance of the matching items in the initial alignment sequence can be considered as 1), the A, B alignment is substantially reliable, so that it can be presumed that the control dependency item within a certain distance A is likely to match the control dependency item within a certain distance B, and the control dependency item within a certain distance A is marked as C ₁ ，C ₂ ，...，C _n The control dependence item within a certain distance of B is D ₁ ，D ₂ ，...，D _n Will (C) _i ，D _j ) (0 < i, j < = n) as a new match, while setting the dependency distance of the newly added match to be 1 more than that of A, B; after several iterations, when the newly generated matching item dependency distance exceeds the set threshold,

the alignment sequence is not increased, so that an extended "one-to-many" alignment sequence is obtained.

c) Screening the alignment sequences of 'one-to-many', and reserving the matching items with high matching probability: in order to measure the matching accuracy of a group of matching items, the following method is introduced for evaluation:

consider the matching term (A _i ，B _j ) Wherein A is _i Representing the ith execution instance of statement A, B _j Represents the j-th execution instance of statement B;

if A _i 、B _j The symbol expressions of (2) are the same, and the matching possible probability value is set to be 0.9; otherwise, the next step of judgment is carried out: if the calculated values of Ai and Bj are different, setting the possible matching probability value to be 0.2; otherwise, the matching accuracy of Ai and Bj cannot be determined, so that a compromise value of 0.5 is given; namely:

if two execution examples of Ai and Bj form a matching item and the code statement A, B of the two execution examples are marked as matching, the probability that Ai and Bj are matched correctly is high, and the matching probability value is set to be 0.9; otherwise, if the code statement A, B corresponding to Ai, bj is marked as matching but Ai, bj is no longer matching after several iterations, setting the matching possible probability value to 0.1; the compromise is set to 0.5 in other cases; namely:

if two execution instances of Ai, bj form a matching item, and the direct dependent items Ax, by of Ai, bj are marked as matching, it can be judged that Ai, bj matching is correct, thus setting the matching possible probability value to 0.99; otherwise, the matching possibility is very little, and the matching possibility probability value is set to be 0.01; namely:

d) Comparing the result of the method with a set threshold value, and adding the result into a new alignment sequence if the result is larger than the threshold value; the steps are repeatedly executed until convergence, so that the final probability-based alignment matching sequence is obtained.

Step 6) the specific process of searching for two key examples of the program is as follows:

a) Traversing the PRINT instances in sequence A, B, and judging whether the values of the parameters are consistent with the number of the parameters: if the types of the parameters of the PRINT examples corresponding to the current two programs are different or the numbers of the parameters are different, directly recording the PRINT examples, wherein the PRINT examples have problems; if the parameters of the PRINT examples corresponding to the current two programs are different, recording the positions of the PRINT examples and the corresponding different parameters.

b) Judging the number of PRINT: if the number of PRINT is consistent and different conditions appear in a), the output of the two programs is consistent, and the program result is correct; if the error program PRINT instances are not consistent, the problem exists in the output, and if the error program PRINT instances are excessive, the correct PRINT instances are printed.

c) Judging the obtained record PRINT example: for the case in step a), the key instance of the two programs is the PRINT instance, and meanwhile, whether the PRINT instance is excessive in the error program needs to be continuously judged, if so, the key instance in the error program is emptied to indicate the special case; for the case in step b), the key instances of the two programs are assigned to corresponding different parameter instances of the corresponding PRINT instance.

d) Judging whether the obtained key instance is a list of one instance, if so, continuing to traverse to find different instances in the list, and then taking the different instances as key instances corresponding to two programs; if the key instance is LDIns, the control dependent item of the key instance needs to be searched, and STIns and BRIns in the control dependent item are used as key instances to replace the original key instance.

The specific process of obtaining the instance related to the key instance in the step 7) is as follows:

a) Firstly, based on the key examples obtained in the step 6), the aligned example sequence is sliced: that is, the dependent item of the key instance is added into the existing key instance, and the finally completed key instance is returned as a slice.

b) The slice of the procedure is traversed and if an instance is in the alignment sequence, an instance of another procedure aligned with the instance is also added to the slice of the other procedure.

Step 8) the specific process of performing the second alignment and obtaining a new slice is as follows:

a) The current slice of both programs (sequence of instances related to the critical instance) is first filtered, and instance dependent items that are not in alignment are deleted.

b) The results of a) are then aligned in sequence, see step 4).

c) Finally, the slices are remade: the slice is first emptied and then traversed to new alignment results, and if aligned but non-matching instances exist, these instances are added to the slice of the corresponding program.

The specific process of marking error information in the step 9) is as follows:

traversing the codes in the error program, marking the behavior error codes if the number of lines of the instances in the error program slice is consistent with the number of lines currently traversed to the codes, and printing the symbol expressions if the statement is an assignment statement.

Claims (2)

1. The code error positioning method based on probability alignment is characterized in that a LLVM (LoW Level Virtual Machine) tool is used for optimizing a source code, extracting symbol information of a program, tracking an execution flow, classifying and storing program instances according to the extracted information, constructing a code structure and a symbol execution flow, aligning the code instances in sequence, carrying out probability alignment on the basis, carrying out secondary alignment according to key instances of two programs and an alignment result slice, focusing on instances which are matched but not matched, and thus positioning errors; the method comprises the following steps:

1) Converting the test code into an intermediate code irrelevant to language by using an LLVM tool, and processing the intermediate code by using Pass to obtain symbol information and execution information of a source program;

2) Analyzing the obtained source program information, classifying and storing the instruction types, and constructing a program structure;

3) Carrying out sequence alignment on an input instance, loading contents in a symbol execution information file, carrying out pairwise matching on slices of a correct version and an error version, carrying out scoring for describing matching probability once according to the current iteration times and execution orders of two items for each matching, wherein a specific scoring rule is that the smaller the difference between the iteration times and the execution times is, the higher the score is, and after initial assignment is finished, integrating matching conditions according to the matching rule, and recording a corresponding item recorded as matching into the sequence alignment to obtain initial sequence alignment; then probability alignment is carried out, alignment items which are generated due to lack of dependency information in initial alignment are deleted, a more accurate alignment result is obtained, namely, all matching items in initial sequence alignment are traversed again, probability initial values are given, the relation between the distance between the two alignment items and a threshold value is judged, and an alignment sequence is expanded; finally, calculating the matching possibility of each group of alignment sequences according to probability rules, respectively adding a certain weight to the results as a result of matching accuracy, comparing the results with a set threshold, if the result exceeds the threshold, recording the result into a new alignment sequence, and executing for multiple times to obtain a probability-based matching sequence;

4) Searching and analyzing the PRINT examples of the two programs in the example sequences of the two programs with the aligned probabilities to obtain key examples of the two programs; slicing the respective key examples obtained in the alignment to obtain examples related to the key examples in the program; and finally, carrying out second alignment on the slices to obtain new slices of the two programs, namely an instance sequence which is matched but not matched, simultaneously obtaining a new alignment result, carrying out each row of printing on codes of the error program, marking the codes containing the slice instances with error information, and simultaneously printing values of symbol expressions.

2. The method for locating code errors based on probability alignment according to claim 1, wherein in the step 2), the structure and execution information of the source program are restored and stored, and an output file obtained after Pass processing of the LLVM is read in; firstly, classifying and storing according to the types of code instances, constructing a code structure, wherein each instance has a unique id identification; and then reading the execution information of the program, processing according to the corresponding code corresponding to the id, recording the execution sequence, and recording the execution sequence into a symbol execution information file.