CN109976806B

CN109976806B - Java statement block clone detection method based on byte code sequence matching

Info

Publication number: CN109976806B
Application number: CN201910003382.6A
Authority: CN
Inventors: 俞东进; 杨加柞; 孙笑笑; 陈信; 王琳
Original assignee: Hangzhou Dianzi University
Current assignee: Hangzhou Dianzi University
Priority date: 2019-01-03
Filing date: 2019-01-03
Publication date: 2022-06-14
Anticipated expiration: 2039-01-03
Also published as: CN109976806A

Abstract

The invention discloses a Java statement block clone detection method based on byte code sequence matching. And finally obtaining similar code segments by adopting a sequence matching and similarity calculation mode. The method accurately extracts the byte code segment at the statement block level by analyzing the execution and the jump of the instruction, and adopts the unique character to represent according to the function realized by the instruction, thereby improving the detection efficiency of code cloning. In the specific process of detecting clone codes, the sequence matching method is applied to a single-character instruction sequence, and compared with other traditional methods, the method has better detection and identification effects.

Description

Java statement block clone detection method based on byte code sequence matching

Technical Field

The invention belongs to the field of software engineering, and particularly relates to a Java statement block code clone detection technology based on byte code sequence matching.

Background

In the software development process, developers often multiplex code by copying-pasting or adding a small amount of modification, and the form of the multiplexed code is code cloning. Studies have shown that the proportion of these cloned codes in software is approximately between 7% and 23%. Code cloning is advantageous on the one hand to reduce the cost of development, but on the other hand it also brings many hazards, for example it introduces bugs in the original code fragments into the system and makes understanding and maintenance of the software difficult. Therefore, the code clone detection can better help people to find the clone code from the software system, thereby providing a basis for software maintenance, code management and the like.

Code clone detection is a hot research area in current software code analysis. In the field of software engineering, code clone detection has many application fields, such as program understanding, code quality analysis, software evolution analysis, error detection and the like, which all need to extract clone code segments in a software system, so that the code clone detection is called as an important and valuable part of the field of software analysis. However, most of the existing code clone detection methods are to identify clone code from the source code of the software system, such as text-based, token-based, abstract syntax tree-based, metric-based, and program dependency graph-based, and the granularity of detection is generally limited to the class and method level. In fact, since similar source code is likely to be compiled into the same bytecode, more accurate results can be achieved from bytecode identification directly. On the other hand, code cloning is more widely present at the statement block level than at the class and method level.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a Java statement block code clone detection method based on byte code sequence matching.

The method comprises the following specific steps:

respectively compiling two different Java source code files into byte codes, further converting the byte codes into byte code text format files, and extracting statement blocks p and q of the byte codes on the basis of the byte code text files;

step (2) extracting instruction sequences from the statement blocks p and q respectively, and representing each instruction by using a unique character according to different functions, thereby forming two single instruction character sequences SCS with the lengths of | p | and | q |, respectively_pAnd SCS_q；

Step (3) for two single character command sequence SCS_pAnd SCS_qConstructing a byte code matching matrix BAM of | p | +1 row and | q | +1 column^p,qAnd initializing BAM^p,qThe matching score of each cell in (a) is 0;

step (4) calculating the byte code matching matrix BAM in sequence from the first row and the first column by row and then by column^p,qMatch score BAM of each cell (i, j) in (b)^p,q[i][j]Is BAM^p,q[i-1][j-1]+σ(SCS_p[i]+SCS_q[j])、BAM^p,q[i-1][j]+σ_Delete、BAM^p,q[i][j-1]+σ_InsertAnd 0; wherein if SCS_pThe ith character and SCS in_qThe j-th character in the sequence is the same, then sigma (SCS)_p[i]+SCS_q[j]) For Match, add cell (i, j) to the starting point set of the closed backtracking path, set it as unvisited, and set the predecessor cell of cell (i, j) according to the following rule: if matching score BAM ^p,q[i][j]Is BAM^p,q[i-1][j-1]+σ(SCS_p[i]+SCS_q[j]) Then the predecessor cell is (i-1, j-1), if the value is BAM^p,q[i-1][j]+σ_DeleteThen the predecessor cell is (i-1, j), if the value is BAM^p,q[i][j-1]+σ_InsertIf the value of the predecessor cell is 0, null is set; when SCS_pThe ith character and SCS in_qIf the j-th character is not the same, then σ (SCS)_p[i]+SCS_q[j]) Is MisMatch, above Match, MisMatch, sigma_DeleteAnd σ_InsertThe values of (A) are respectively 2, -2 and-2;

selecting a cell with the highest matching score from all cells which are not accessed in a starting point set of a closed backtracking path, sequentially selecting precursor cells of the cell from the cell until the cell reaches a null cell of the precursor cells, thereby obtaining a closed backtracking path, setting each cell in the closed backtracking path as being accessed, and respectively and sequentially selecting row coordinates and column coordinates of each cell in the closed backtracking path to form two single character instruction sequences suspected to be cloned;

step (6) repeating step(5) To obtain SCS_pAnd SCS_qThe single character instruction subsequence pairs of all suspected clones are combined to form two three-dimensional vectors

Wherein p is₁As SCS_pMiddle, and SCS_qThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L _pIs SCS_pCorresponding unicode instruction sequence length, q₁As SCS_qMiddle, and SCS_pThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L_qIs SCS_qThe length of the corresponding single character instruction sequence; and calculating cosine similarity of the two three-dimensional vectors, namely cosine similarity of p and q, and if the cosine similarity is greater than a certain threshold, considering the statement blocks p and q as cloned byte code segments and mapping the cloned byte code segments back to the source code.

The Java statement block code clone detection method based on byte code sequence matching provided by the invention comprises a group of modules, and the modules comprise: the system comprises a code preprocessing module, a feature extraction and normalization module and a code clone detection module.

The code preprocessing module is used for extracting byte code blocks, firstly, an Oracle JDK and a batch compiler are adopted to carry out batch compilation on a source code file and convert the source code file into a text form, and then the byte code blocks are extracted according to the execution and jump conditions of instructions in the byte code file.

The characteristic extraction module extracts an instruction sequence from the byte code block, and each instruction is represented by adopting a unique character according to different functions.

The code clone detection module detects two single character instruction sequences SCS corresponding to any two byte code blocks p and q _pAnd SCS_qAs input, two single character command sequences SCS_pAnd SCS_qConstruction of bytecode matching matrix BAM^p,qCalculating the matching score of each cell of the byte code matching matrix, constructing a closed backtracking path, acquiring similar subsequences, and calculating the distance between two byte code blocksAnd finally obtaining similar byte code blocks according to the similarity.

The method provided by the invention extracts the byte code segment at the statement block level by analyzing the execution and the jump of the instruction, then extracts the instruction from the byte code segment and adopts unique characters to represent according to different realized functions. And finally obtaining similar code segments by adopting a sequence matching and similarity calculation mode.

The method accurately extracts the byte code segment at the statement block level by analyzing the execution and the jump of the instruction, and adopts the unique character to represent according to the function realized by the instruction, thereby improving the detection efficiency of code cloning. In the specific process of detecting clone codes, the sequence matching method is applied to a single-character instruction sequence, and compared with other traditional methods, the method has better detection and identification effects.

Drawings

FIG. 1 is a workflow diagram of code clone detection;

FIG. 2 is a diagram of an example extraction of a byte code block;

fig. 3 is a flow diagram of byte code sequence matching.

Detailed Description

The specific implementation of the code clone detection method of multiple Java statement blocks based on byte code sequence matching mainly comprises 3 steps (as shown in FIGS. 1 and 3):

(1) according to two input Java files, an Oracle JDK and a batch compiler are adopted to compile a source code into a byte code file and further convert the byte code file into a text form file, and code segments at a statement block level are extracted from the byte code file according to the execution and jump conditions of instructions; (2) extracting an instruction sequence required by the method from the bytecode code segment at the statement block level, and expressing each instruction in the instruction sequence by adopting a unique character according to the realized function so as to form a single-character instruction sequence; (3) in the code clone detection stage, the single character instruction sequence is utilized to carry out sequence matching and similarity calculation between byte code fragments, and the similarity calculation is mapped back to the source code.

For convenience of description, the associated symbols are defined as follows:

IS_BCF: instruction sequence in the Byte code fragment BCF No, denoted IS _BCF＝(I₁,I₂,...,I_i,...,I_n) In which I_iRepresents the ith (1 ≦ i ≦ n) instruction in the byte code fragment BCF.

SCS_BCF: single character instruction sequence in byte code fragment BCF, denoted SCS_BCF＝(S₁,S₂,...,S_i,...,S_n) In which S is_iIndicates the I (I ═ 1, 2.., n) th unicode, which is instruction I_iUnique characters are employed according to the function implemented.

SLP_BCF: i of a single character instruction sequence in a byte code fragment BCF^thLeft prefix sequence, SCS for a single character sequence_BCF＝(S₁,S₂,...,S_i,...,S_n) Its left prefix sequence is SLP_BCF＝(S₁,S₂,...,S_i)(1≤i≤n)。

BAM^p,q: the bytecode matches the matrix. The cell scores in the ith row and the jth column in the matrix represent the matching scores of the ith row before the byte code segment p and the jth row before the byte code segment q. In particular, for a single character sequence SCS in two byte code segments p, q_pAnd SCS_q，BAM^p,q[i][j]Representation of SCS_pLeft prefix SLP of_i ^pAnd SCS_qLeft prefix of

The match between them.

preCell: is the predecessor cell from which the cell (i, j) score originated.

(1) Code preprocessing of source code files

In order to detect code cloning on a bytecode code segment at a Java statement block level, a Java source code is compiled into a bytecode file through Oracle JDK and a batch compiler and converted into a text form file, and then the bytecode code segment at the statement block level is extracted according to the execution and jump conditions of instructions.

When extracting the bytecode code block at the statement block level, we analyze the control transfer instruction. If the instruction is a goto instruction, we will extract the byte code fragment between the next instruction of the instruction to the jump number instruction. If the instruction is an if-related instruction, comparing the current line number with the jump line number of the instruction, if the current line number is smaller than the jump line number, directly extracting a byte code segment between the two numbers, otherwise, extracting a byte code segment between the current line number and an instruction which is previous to the instruction and corresponds to the jump line number, and deleting a code segment of which the current line number is consistent with the jump line number of the instruction from a corresponding code segment extracted by a previous goto instruction. If the instruction is a switch instruction, we will get the largest number in its jump line number set and extract the byte code fragments from the current line number to the largest number, the whole process is shown in FIG. 2.

(2) Extracting instruction features from byte code fragments

After obtaining the bytecode fragment BCF, the instruction sequence IS IS extracted_BCF＝(I₁,I₂,...,I_i,...,I_n) For instruction sequence IS_BCFIn other words, single character instruction sequence SCS is adopted _BCF＝(S₁,S₂,...,S_i,...,S_n) To show that we use all lower case letters and ASCII characters as single characters.

(3) Sequence matching and similarity calculation

First, a single character sequence SCS is generated based on two byte code segments p and q_pAnd SCS_qConstruction of BAM^p,qAnd initialized to 0, and then the value of each cell in the matching matrix is calculated by formula 1. Wherein,

is the score of the cell (i, j) in the bytecode matching matrix. If two single character sequences SCS_pAnd SCS_qThe ith ofCharacter and j-th character are the same, then σ (SCS)_p[i]+SCS_q[j]) If the value of (1) is Match, otherwise, is MisMatch, and the cell is added into the starting point set of the closed backtracking path and is set to be not accessed. Match, MisMatch, sigma_DeleteAnd σ_InsertRespectively takes values of 2, -2 and-2

Then, look for the predecessor cell preCell through equation 2, and select the highest scoring cell among all the unvisited cells from the set of close backtracking starting points. Starting from the unit, the precursor cells preCell are continuously traced until reaching the cells with the matching score of 0, then the row coordinates and the column coordinates of each cell in the closed tracing path are respectively and sequentially selected to form two single character instruction sequences suspected to be cloned

Repeating the above process to obtain SCS_pAnd SCS_qThe single character instruction subsequence pairs of all suspected clones are combined to form two three-dimensional vectors

Wherein p is₁As SCS_pMiddle, and SCS_qThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L_pIs SCS_pCorresponding unicode instruction sequence length, q₁As SCS_qMiddle, and SCS_pThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L_qIs SCS_qThe length of the corresponding single character instruction sequence; calculating cosine similarity of two three-dimensional vectors, namely cosine similarity of p and q, if the cosine similarity is greater than a certain threshold value, regarding the statement blocks p and q as cloned byte code segments, and mapping the cloned byte code segments back to the source generationAnd (4) code.

Claims

1. A Java statement block clone detection method based on byte code sequence matching is characterized by comprising the following specific steps:

step 1, compiling two different Java source code files into byte codes respectively, further converting the byte codes into byte code text format files, and extracting statement blocks p and q of the byte codes on the basis of the byte code text files;

step 2, extracting instruction sequences from the statement blocks p and q respectively, representing each instruction by using a unique character according to different functions, thereby forming two single instruction character sequences SCS with the lengths of | p | and | q |, respectively _pAnd SCS_q；

Step 3. for two single character command sequences SCS_pAnd SCS_qConstructing a byte code matching matrix BAM of | p | +1 row and | q | +1 column^p,qAnd initializing BAM^p,qThe matching score of each cell in (a) is 0;

step 4, calculating byte code matching matrix BAM in sequence from first row and first column by row and then by column^p,qMatch score BAM of each cell (i, j) in (b)^p,q[i][j]Is BAM^p,q[i-1][j-1]+σ(SCS_p[i]，SCS_q[j])、BAM^p,q[i-1][j]+σ_Delete、BAM^p,q[i][j-1]+σ_InsertAnd 0, the maximum of the four values; wherein if SCS_pThe ith character and SCS in_qThe j-th character in the sequence is the same, then sigma (SCS)_p[i],SCS_q[j]) For Match, adding the cell (i, j) into the starting point set of the closed backtracking path, setting the starting point set as unvisited, and then setting the predecessor cell of the cell (i, j) according to the following rules: if matching score BAM^p,q[i][j]Is BAM^p,q[i-1][j-1]+σ(SCS_p[i]，SCS_q[j]) Then the predecessor cell is (i-1, j-1), if the value is BAM^p,q[i-1][j]+σ_DeleteThen the predecessor cell is (i-1, j), if the value is BAM^p,q[i][j-1]+σ_InsertIf the value of the precursor cell is 0, null is set; when SCS_pThe ith character and SCS in_qIf the j-th character is not the same, then σ (SCS)_p[i],SCS_q[j]) Is MisMatch, above Match, MisMatch, sigma_DeleteAnd σ_InsertThe values of (A) are respectively 2, -2 and-2;

step 5, selecting the cell with the highest matching score from all the unvisited cells in the starting point set of the closed backtracking path, sequentially selecting the precursor cells of the closed backtracking path from the cell until the cell with null precursor cells is reached, thereby obtaining a closed backtracking path, setting each cell in the closed backtracking path as visited, and respectively and sequentially selecting the row coordinate and the column coordinate of each cell in the closed backtracking path to form two single character instruction sequences suspected to be cloned;

Step 6, repeating the step 5 to obtain SCS_pAnd SCS_qAll the single character instruction subsequence pairs of suspected clone in the Chinese character library are combined, and two three-dimensional vectors are formed

Wherein p is₁As SCS_pMiddle, and SCS_qThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L_pIs SCS_pCorresponding unicode instruction sequence length, q₁As SCS_qMiddle, and SCS_pThe subsequences are the sum of the lengths of all single character instruction subsequences of suspected clone pairs, L_qIs SCS_qThe length of the corresponding single character instruction sequence; and calculating cosine similarity of the two three-dimensional vectors, namely cosine similarity of p and q, and if the cosine similarity is greater than a certain threshold, considering the statement blocks p and q as cloned byte code segments and mapping the cloned byte code segments back to the source code.