JP5342407B2 - Program analysis method, program analysis program, and program analysis apparatus - Google Patents

Description

  The present invention relates to a program analysis method, a program analysis program, and a program analysis apparatus.

  In a large-scale program system composed of a large number of program codes, a highly reliable program system in the past is used to increase or decrease the equipment to be controlled and to realize the requirements from the user. In many cases, a new program system is developed by applying some modifications and corrections to the program code. Adopting this development method will divert program system A with a proven track record and newly develop program system B. In such a case, these two program systems become huge clones.

Further, even in a large-scale program system A, there are many cases in which similar program code that appears many times in the program code exists. Such a program code appearing in one program system is also called a clone. This is caused by copying (copying) and pasting (pasting) the program code. A large-scale program system realized by such a development method has many similar program codes (clones), which causes a significant deterioration in understanding and decipherability.
For example, since hardware equipped with a dedicated program has become obsolete, when replacing with new hardware, the installed program is also obsolete, so a program with a new architecture needs to be developed. In such a case, it is often performed to develop a new program by diverting an old-style program, but bugs may occur due to differences in handling of global variables.

  In the maintenance / maintenance of a program, when a defect is found and corrected in a certain part of the program code, it is necessary to examine whether a similar correction is necessary for a part similar to that part. However, in many cases, the program code has been modified to a greater or lesser extent. By simple character string search, etc., it is comprehensive to see where such a part exists in a large amount of program code. It is difficult to grasp.

Furthermore, since control systems that support many social infrastructures are required to have absolute quality, hardware is often composed of devices with a long life, environmental resistance, and high reliability. For this reason, there are also many program systems that have been maintained and modified for more than 10 years for the program systems installed in such hardware. However, in recent years, it has become difficult to procure replacement parts such as semiconductors and hard disks even for program systems that have been operating on special hardware for more than 10 years. It has become. Even in this case, similar code that appears many times in the program code becomes a problem that hinders development efficiency and reliability for the reasons described above.
In addition, when a program system is migrated to another system, there is no guarantee that the update order is the same as the development order of the legacy system, so it may be unclear which is the clone.
In such a case, the clone relationship of the program code is not analyzed only for the program code that constitutes the update system, but the development history, history, and transition in the past program system are clarified. If this is not clarified, it will cause problems that hinder development efficiency and reliability.

As such software for analyzing an existing program code clone, CCFinder is disclosed (for example, see Non-Patent Document 1). In CCFinder, the lexical analysis is performed on the program code, the token is converted into a specific character string, and collation is performed. The variable name and the function (subroutine) name to be called in the program code It is possible to detect the similarity of the program code even if is changed.
In Patent Document 1, the program code is standardized, each token string is created, a correlation matrix is created from each token, the correlation matrix is regarded as a binarized image, and a texture amount is calculated. A similarity evaluation program, a similarity evaluation apparatus, and a similarity evaluation method for calculating a similarity between program codes by calculating a distance between reference vectors corresponding to a matrix are disclosed.

JP 2008-46695 A

Toshihiro Kamiya, CCFinder homepage, [online], [October 2, 2009 search], Internet <URL: http://www.ccfinder.net/ccfinderx-j.html>

The program code clone detection tool (CCFinder) described in Non-Patent Document 1 and the technique described in Patent Document 1 are methods for performing lexical analysis and analyzing program code clones by calculation processing according to a predefined rule. . In this method, it is possible to extract clones, but innumerable clones are analyzed in a large-scale program system, so, for example, if you want to classify as a different function because the variables used are different, etc. In many cases, it was difficult to select the clone information that was really necessary, and it was not enough to meet the requirements.
Furthermore, the program code clone detection tool described in Non-Patent Document 1 lexically analyzes a program and directly compares the similarity of program syntax. There is a problem that even if it should be determined that is different, it is determined as a clone. In other words, even if a variable uses the same string, for example, if one is a global variable and the other is a local variable, the functions that use these variables have essentially different functions. Nevertheless, it is regarded as the same function.
Such a problem is not solved even in Patent Document 1.

  The present invention has been made in view of such a background, and an object of the present invention is to accurately display clone lines.

In order to solve the above-described problem, the present invention provides a parent-child having a modification source program code as a parent among a plurality of program codes and a modification-destination program code modified by adding or correcting a token as a child. A program analysis method by a program analysis apparatus for deriving and displaying a relation, wherein the program analysis apparatus has a function dictionary in which functions that can be regarded as the same function are registered, and variables used in the function , although the description format is different memory access dictionary having registered the kind, have been stored in the storage unit, the program analyzing device, (a1) a plurality of program code to be analyzed, performs decompose lexical analysis on the tokens, (a2 ) together is generated based on the results of the lexical analysis in a given said program code stored in the storage unit Wherein the template token in a given program code is registered, the result of each lexical analysis in each program codes, wherein a function dictionary, with reference to a result of comparison with the token that is registered in the template description formats differ performs processing to determine as the same function function which can be regarded as the same function in the function dictionary for each program code, (a3) and the template, the result of the lexical analysis in each of the program code, wherein Referring to a memory access dictionary a result of comparison with the functions registered in the template, but the format is the same function, it is determined that different functions functions use different types of variables, the same perform the narrowing down untreated functions for each program code, (a4) said different Seki And the determined function different token related tokens as being, among the plurality of program codes for the two program code, taken ordinate the occurrence position of the token in one of the program code, the appearance of the other program code tokens position represented by the horizontal axis, when the same token appears in the two program code, and the vertical axis corresponding to the coordinate of the horizontal axis corresponding to the occurrence position of the same token x, the occurrence position of the same token When the coordinate of y is y, clone analysis is performed to generate data plotted at the point (x, y). (A5) Based on the result of the clone analysis , a parent-child relationship between the template and the program code is derived. (A6) The processing of (a4) and (a5) is performed using the template and program to be analyzed. (A7) A system diagram in which the template and the program code from which the parent-child relationship is derived is connected by a line is displayed on the display unit.
Other solutions are described in the embodiments.

  According to the present invention, clone lines can be accurately displayed.

It is a functional block diagram which shows the structural example of the program analysis apparatus which concerns on this embodiment. It is a figure which shows the outline | summary of the program analysis process which concerns on this embodiment. It is a flowchart which shows the procedure of the program analysis process which concerns on this embodiment. It is a figure for demonstrating the lexical analysis process in step S102. It is a figure for demonstrating the template creation process in step S104. It is a figure for demonstrating the function dictionary creation process in step S105. It is a figure for demonstrating the memory access dictionary creation process in step S106. It is a figure for demonstrating the similarity calculation process in step S107. It is a figure for demonstrating the classification result creation process in step S110 (the 1). It is a figure for demonstrating the classification result creation process in step S110 (the 1). It is a clone analysis figure for demonstrating the clone analysis process in step S111. It is a figure for demonstrating the part of parent-child relationship derivation processing among the parent-child relationship derivation / derivation rate calculation processing of step S112. It is an example of the clone analysis figure displayed on a result display part after a clone analysis process is complete | finished. It is a figure which shows the example of the systematic diagram for demonstrating the systematic diagram creation process in step S114. It is a figure which shows the specific example of a template, a function dictionary, and a memory access dictionary. It is a figure which shows the specific example of a template and a program code. It is a figure which shows the similarity result with respect to the template a of each program code which comprises the program code group (alpha). It is a figure which shows the systematic diagram of program code group (alpha) and (beta) (the 1). It is a figure which shows the systematic diagram of program code group (alpha) and (beta) (the 2).

Next, modes for carrying out the present invention (referred to as “embodiments”) will be described in detail with reference to the drawings as appropriate. In the following description, a program code group is a program that functions by a plurality of program codes being aggregated. For example, if one program is a program code group, the program code is a main function or a sub function. Alternatively, the process may be program code.
Note that, in each drawing of the present embodiment, the same reference numerals are given to the same components, and description thereof is omitted.

"Device configuration"
FIG. 1 is a functional block diagram illustrating a configuration example of the program analysis apparatus according to the present embodiment.
The program analysis apparatus 1 includes a control unit 100 that controls the operation of the apparatus, a storage unit 200 that stores information, a result display unit (display unit) 300 such as a display, and a user interface unit 400 such as a keyboard and a mouse. Have.
The control unit 100 includes an analysis target program code group registration unit 101, a lexical analysis unit 102, a template creation unit 103, a function dictionary creation unit 104, a memory access dictionary creation unit 105, a similarity calculation unit 106, and a similarity A degree analysis result creation unit 107, a classification result creation unit 108, a clone analysis unit 109, and a system diagram creation unit 110.

The analysis target program code group registration unit 101 has a function of performing processing for registering the program code group 201 including a plurality of program codes to be analyzed via the user interface unit 400.
The lexical analysis unit 102 has a function of performing a lexical analysis process that decomposes the registered program code into tokens. Note that a token is a minimum unit of words and symbols constituting a program code such as a function.
The template creation unit 103 has a function of performing processing for creating the template 202 based on the result of lexical analysis (lexical analysis result 205), information input from the user interface unit 400, and the like. Details of the template 202 will be described later.
The function dictionary creation unit 104 has a function of creating a function dictionary (token dictionary) 203 in which modified examples of functions are registered based on the template 202, the lexical analysis result 205, information input from the user interface unit 400, and the like. Have Details of the function dictionary 203 will be described later.
The memory access dictionary creation unit 105 has a function of creating the memory access dictionary 204 in which the types of variables referred to by functions are registered based on the template 202, information input from the user interface unit 400, and the like. Details of the memory access dictionary 204 will be described later.

The similarity calculation unit 106 has a function of calculating the similarity by comparing the structure of the program code in the template 202 with the structure of the registered program code.
The similarity result creation unit 107 has a function of causing the result display unit 300 to display the result output by the similarity calculation unit 106.
The classification result creation unit 108 has a function of causing the result display unit 300 to display the similarity result created for each template 202 as a classification result.
The clone analysis unit 109 has a function of performing clone analysis based on the lexical analysis result 205, the function dictionary 203, and the memory access dictionary 204, and displaying the result of the clone analysis on the result display unit 300.
System diagram creation unit 110, based on the result of clonal analysis out guide the parent-child relationship between program code, having a parent-child relationship between program code function to be displayed on the result display section 300 as a system diagram.
Note that the control unit 100 and each of the units 101 to 110 have a program analysis program stored in a ROM (Read Only Memory) or HD (Hard Disk) (not shown) developed in a RAM (Random Access Memory) and a CPU (Central Processing Memory). It is embodied by being executed by (Unit).

  The storage unit 200 stores each information of a registered program code group 201, a lexical analysis result 205, a template 202, a function dictionary 203, a memory access dictionary 204, and a token registration determination table 206. As described above, the lexical analysis result 205, the template 202, the function dictionary 203, the memory access dictionary 204, and the token registration determination table 206 will be described later.

《Outline of processing contents》
FIG. 2 is a diagram showing an outline of the program analysis processing according to the present embodiment.
First, it is assumed that program codes a1, a2, a3, and a4 are registered as the program code group α, and program codes b1, b2, and b3 are registered as the program code group β.
First, the program analysis apparatus 1 calculates the similarity between program codes in each program code group 201 using the template 202, the function dictionary 203, the memory access dictionary 204, and the like. That is, the similarity between the program codes a1, a2, a3, and a4 is calculated, and the similarity between the program codes b1, b2, and b3 is calculated (S1).
Next, the program analysis apparatus 1 performs a clone analysis process (S2), calculates a derivation rate based on the created clone analysis diagram, and derives a parent-child relationship between program codes (S3).
Then, the program analysis apparatus 1 displays the parent-child relationship between the program codes on the result display unit 300 as a system diagram (S4). Here, the parent-child relationship means that the original program code is a parent, and a token is added to the parent program code, and editing such as correction is made a child.
Note that “a” in the system diagram of FIG.

<Processing procedure>
Next, the procedure of the program analysis process according to this embodiment will be described with reference to FIGS.
First, in the flowchart of FIG. 3, the flow of the processing procedure will be described, and details of each processing will be described with reference to FIGS.
FIG. 3 is a flowchart showing the procedure of the program analysis process according to the present embodiment.
First, a plurality of analysis target program codes are input to the storage unit 200 via the user interface unit 400. Thereafter, an analysis target program code registration process is performed in which a program code to be analyzed is selected and registered from among the program codes input to the storage unit 200 via the user interface unit 400 (S101).
Next, in the lexical analysis process, one of the program codes selected in step S101 is selected, information that is not directly related such as a comment part is removed from the program code, and divided in token units. A lexical analysis process is performed (S102).
Then, the control unit 100 determines whether or not the lexical analysis processing has been completed for all the program codes selected in step S101 (S103).
When the lexical analysis processing has not been completed for all program codes as a result of step S103 (S103 → No), the control unit 100 returns to the processing of step S102, and the lexical analysis processing is performed for the program code for which lexical analysis processing has not been performed. I do.

As a result of step S103, when the lexical analysis processing has been completed for all program codes (S103 → Yes), the template creation unit 103 uses the lexical analysis result 205 and the information input via the user interface unit 400 as a basis. A template creation process for creating a template 202 in which the tokens are weighted is performed (S104).
Then, the function dictionary creation unit 104 performs a function dictionary creation process for creating the function dictionary 203 based on the lexical analysis result 205, the template 202, and information input via the user interface unit 400 (S105).
Next, the memory access dictionary creation unit 105 performs a memory access dictionary creation process for creating the memory access dictionary 204 based on the template 202 and information input via the user interface unit 400 (S106).
A plurality of different types of templates 202 may be produced. When creating a plurality of templates 202, the processes in steps S104 to S106 are repeated.

Then, the similarity calculation unit 106 performs a similarity calculation process for calculating the similarity based on the template 202 and the token registration determination table 206 created from the template 202 (S107).
Next, the control unit 100 determines whether or not the similarity calculation processing has been completed for all program codes for the template 202 (S108).
As a result of step S108, when the similarity calculation processing has not been completed for all program codes (S108 → No), the control unit 100 returns the processing to step S107, and the similarity calculation unit 106 has not calculated the similarity. The similarity is calculated for the program code.

If the similarity calculation processing has been completed for all program codes as a result of step S108 (S108 → Yes), the similarity result creation unit 107 causes the result display unit 300 to display the result of the similarity calculation processing. Display processing is performed (S109).
Next, the classification result creation unit 108 performs a classification result creation process for classifying the program code for each template 202 by excluding the program codes having a similarity equal to or less than the threshold from the processing target (S110).
Then, the clone analysis unit 109 performs clone analysis on each combination of the template 202 and the program code based on the template 202, the lexical analysis result 205, the function dictionary 203, and the memory access dictionary 204, and the result is displayed in the result display unit 300. The clone analysis process to be displayed is performed (S111).

Next, the system diagram creation unit 110 performs parent-child relationship derivation / derivation rate calculation processing for deriving a parent-child relationship between templates and program codes and calculating a derivation rate based on the clone analysis result (S112).
Then, the control unit 100 determines whether or not the parent-child relationship derivation / derivation rate calculation processing is completed for all combinations of program codes (S113).
As a result of step S113, when the parent-child relationship derivation / derivation rate calculation processing is not completed for all combinations (S113 → No), the control unit 100 returns to the processing of step S112, and the system diagram creation unit 110 calculates the derivation rate. Parent-child relationship derivation / derivation rate calculation processing is performed for combinations of program codes that have not been performed.
If the parent-child relationship derivation / derivation rate calculation processing is completed for all combinations as a result of step S113 (S113 → Yes), a system diagram showing the parent-child relationship between program codes is created and displayed on the result display unit 300 A system diagram creation process is performed (S114).

(S102: Lexical analysis processing)
Hereinafter, the main processing will be described in detail with reference to FIGS. 4 to 14 with reference to FIGS. 1 and 3.
FIG. 4 is a diagram for explaining the lexical analysis processing in step S102.
Here, the lexical analysis refers to a process of removing information that is not necessary for directly comparing the similarity, such as a comment part, from the program code, and further dividing the program code in units of tokens. First, when the lexical analyzer 102 reads the program code selected / registered in step S101, the lexical analyzer performs the following lexical analysis process.
For example, when the program code is written in C language, the character string following “//” in the read program code and the comment sentence enclosed by “/ *” and “* /” are deleted. Furthermore, the program code from which the comment is deleted is divided into tokens.
FIG. 4 shows a specific example of the program code and the token string subjected to lexical analysis according to the above procedure.

A lexical analysis result of the program code 401 is a lexical analysis program code 402, and a lexical analysis result of the program code 411 is a lexical analysis program code 412. Even between program codes that seem to have low mutual similarity at first glance, it is possible to accurately calculate the similarity by lexical analysis of the program code in this way.
The lexical analysis process is a known technique.

(S104: Template creation process)
FIG. 5 is a diagram for explaining the template creation processing in step S104.
In the template 202, the result of the lexical analysis in step S102 is displayed on the result display unit 300, and the user inputs information via the user interface 400 with reference to the displayed lexical analysis result 205 (FIG. 1). Created by.
For example, the template 202 is created from the program code 501 shown in FIG.
As shown in FIG. 5, in the template 202, the structure of the program code is described for each token by using a function, a system call, and the like described in the processing order in the program code. For example, if the program code 501 is C language, tokens such as “if” sentence, “else” sentence, “for” sentence, common function, and system call are registered in the template 202.
Here, the token registered in the template 202 is not limited to all tokens in the program code, but may be any token unique to the program code.

Furthermore, as shown in FIG. 5, the user can weight each token registered in the template 202. That is, it is possible to define a characteristic structure of the template 202 by applying strong weighting to a particularly important token in the template 202.
A plurality of templates 202 can be created, for example, a template 202 created based on the program code A, a template 202 created based on the program code B, and the like. For example, by using the template 202 created based on the program code A, it can be determined whether or not it is close to the program code A. By using the template 202 created based on the program code B, the program code B It can be determined whether it is close to.

  By creating the function dictionary 203 described later in FIG. 6, it is possible to handle a plurality of different functions as a common definition. In other words, the function (or token) defined as the same kind of function in the function dictionary 203 is assumed to be the same kind of function, and the similarity can be increased. In addition, for tokens for which assignment statements and memory are copied, the attribute of the copy destination variable can be registered in the memory access dictionary 204 described later in FIG. 7, and the copy destination can be a global variable or a local variable. It is possible to define something.

(S105: Function dictionary creation process)
FIG. 6 is a diagram for explaining the function dictionary creation processing in step S105.
The user creates a function dictionary 203 by inputting information through the user interface 400 with reference to the template 202 and the lexical analysis result 205 (FIG. 1) displayed on the result display unit 300.
As shown in FIG. 6, for example, common functions having substantially the same meaning (that is, functions created using a copy or the like) “sub_A1 ()” and “sub_A2 ()” are the same as “funcA ()” in the template 202. A function dictionary 203 is defined as a group function.
By doing so, it is possible to determine that similar program codes differ only in the calling common function when calculating the similarity described later. That is, even if they are different functions, functions registered as functions of the same group in the function dictionary 203 can be the same function.
The function registered in the function dictionary 203 may be a function registered in the template 202.

(S106: Memory access dictionary creation process)
FIG. 7 is a diagram for explaining the memory access dictionary creation processing in step S106.
As shown in FIG. 7, for each variable used in the token defined in the template 202, the user determines whether the variable is a global variable, a local variable, or a plurality of global variables that are a plurality of global variables. The memory access dictionary 204 is created by inputting such variable types via the user interface 400.
By doing this, when referring to similarity, which will be described later, functions that refer to different types of variables, such as global variables and local variables that are referenced in the same format but are referenced, are distinguished as different functions. can do.
The memory access dictionary 204 is created by the user with reference to the template 202 displayed on the result display unit 300 and the lexical analysis result 205.

Although the description contents of the template 202, the function dictionary 203, and the memory access dictionary 204 are arbitrary, it is desirable to define the characteristic structure of the clone code to be extracted. These can be created once and then stored in the storage unit 200, and can be read and edited any number of times.
Further, the template 202, the function dictionary 203, and the memory access dictionary 204 may be created in advance before the processing of FIG.

(S107: similarity calculation processing)
FIG. 8 is a diagram for explaining the similarity calculation processing in step S107.
First, the similarity calculation unit 106 creates a token registration determination table 206 as shown in FIG. 8 for each program code.
First, the similarity calculation unit 106 copies the template 202 created in step S104 and compares it with the lexical analysis result 205 created in step S102, thereby registering it in the template 202 in the program code to be processed. It is determined whether or not there is a token that is present. If there is a token, “◯” is registered in the corresponding registration determination column, and if not, “X” is registered.
At this time, the similarity calculation unit 106 refers to the function dictionary 203 or the memory access dictionary 204, and if there is a token of a different format, it is registered in the function dictionary 203. If “◯” is registered in the column and a different variable is referenced in the memory access dictionary 204 even if the token has the same format, “X” is registered in the registration determination column of the token registration determination table 206.

  This token registration determination table 206 is created for each program code. For example, if there are a template A and a template B, and there is a program code A to a program code D, a token registration determination table 206 of the program code A to the program code D based on the template A is created. Token registration determination table 206 of program code A to program code D based on the above is created.

Next, the similarity calculation unit 106 adds the weights of the tokens for which “◯” is registered in the token registration determination table 206 of each program code, and the addition result is a value obtained by adding all the weights (maximum score). ) And then multiply by 100 to calculate the similarity.
That is, the similarity calculation unit 106 calculates the similarity from the following equation (1).

Similarity (%) = program code score / maximum score × 100 (1)

For example, the similarity obtained from the token registration determination table 206 of the program code D shown in FIG. 8 is (1 + 2 + 1) ÷ (1 + 1 + 2 + 1 + 1) × 100 = 66 (%).
The similarity calculation unit 106 calculates the similarity for all program codes for the template 202 that has been created.

  Note that the template 202 and the threshold value that are the processing target of similarity calculation may be designated by the user. The designated timing may be any time before the similarity calculation process.

(S110: Classification result creation process)
9 and 10 are diagrams for explaining the classification result creation processing in step S110.
The classification result creation unit 108 registers program codes having a similarity equal to or higher than a preset threshold as a group.
In the example of FIG. 9, since the threshold is set to “30”, the program code C having a similarity of “20 (%)” is excluded from the group.

In addition, since a plurality of templates 202 can be created as described above, grouping by different templates 202 as shown in FIG. 10 is also possible.
In the example of FIG. 10, two groups of program codes A, B, and D grouped based on the template A and program codes A, C, and D grouped based on the template B are created. .

  The classification result creation unit 108 causes the result display unit 300 to display a group of program codes as shown in FIGS.

(S111: Clone analysis process)
FIG. 11 is a clone analysis diagram for explaining the clone analysis processing in step S111.
The clone analysis unit 109 performs clone analysis on the program code in the following procedure.
First, the clone analysis unit 109 selects a template 202 and a program code pair or a program code and a program code pair from the grouped program codes as shown in FIG. A clone analysis diagram 1101 is created in which the code token appearance position and the horizontal axis are the other template 202 or the program code token appearance position.
The clone analysis unit 109 plots the same token on the clone analysis diagram 1101 on the basis of the template 202 to be compared and the lexical analysis result 205 of the program code.
That is, for the two program codes, the clone analysis unit 109 takes the appearance position of the token of one program code on the vertical axis and the appearance position of the token of the other program code on the horizontal axis. When appearing in one program code, the coordinate of the horizontal axis corresponding to the appearance position of the same token is x, the coordinate of the vertical axis corresponding to the appearance position of the same token is y, and the point of (x, y) A clonal analysis diagram is created by plotting.
At this time, the clone analysis unit 109 stores the function registered in the function dictionary 203 even if the format is different, or the function referring to the same type of variable in the memory access dictionary 204 as the same token in the clone analysis diagram 1101. Plot. On the contrary, even if the format is the same, the clone analysis unit 109 sets different functions referring to different variables in the memory access dictionary 204, and does not perform plotting.

As shown in FIG. 11, in the clone analysis diagram 1101, among the plot portions between token strings determined to be the same, a portion that is a long straight line is a common token portion between program codes. This straight portion is referred to as a clone piece. If the two program codes are the same, a continuous straight line is plotted from the origin to the lower right.
In the clone analysis diagram 1101, a plurality of points plotted in the horizontal direction and the vertical direction, that is, a place where a plurality of other token appearance positions correspond to one token appearance position are the function dictionary 203, The memory access dictionary 204 determines that one token in one program code is identical to a plurality of tokens in the other program code.

(S112: Parent-child relationship derivation process)
FIG. 12 is a diagram for explaining a part of the parent-child relationship derivation process in the parent-child relationship derivation / derivation rate calculation process in step S112.
FIG. 12 shows that when a conditional statement is inserted in the middle of the program code A, a discontinuous portion (broken arrow) is generated in the clone analysis diagram 1201. Thus, in the clone analysis diagram 1201, when the clone pieces are discontinuous, it is considered that the processing unique to the program code is described in that line.
That is, if the original program code is program code A and the program code A is additionally inserted with a conditional statement as program code A1, then in the clone analysis diagram 1201 of FIG. It is discontinuous in a state shifted in the horizontal right direction. Thus, if the clone piece is shifted in the horizontal right direction, the program code indicated by the token appearance position on the horizontal axis (here, program code A1) is the program code indicated by the token appearance position on the vertical axis (here, It can be seen that an additional token is inserted into the program code A).
On the other hand, if the program code A is obtained by additionally inserting a token into the program code A1, a discontinuous portion in which the clone pieces are shifted in the vertically downward direction is generated in the clone analysis diagram.

(S112: Parent-child relationship derivation / derivation rate calculation processing)
FIG. 13 is an example of a clone analysis diagram displayed on the result display unit after the clone analysis process is completed.
In FIG. 13, template A-program code A (clone analysis diagram 1301), template A-program code B (clone analysis diagram 1302), template A-program code D (clone analysis diagram 1303), program code A-program code B (Clone analysis diagram 1304), a clone analysis diagram for a combination of program code A-program code D (clone analysis diagram 1305), program code B-program code D (clone analysis diagram 1306),. Here, there is no combination including the program code C because, as shown in FIG. 9, since the similarity of the program code C is equal to or less than the threshold value, an example excluded from the group is shown.

Here, the template A is the template 202 (FIG. 1) created based on the program code A. Although it is based on the program code A, it is not completely the same. Therefore, a discontinuous portion is generated in the clone analysis diagram of template A-program code A. In FIG. 13, discontinuous portions are indicated by broken-line arrows.
The program code A is an original program code, and the program code B is a program code obtained by adding processing to the program code A. The program code D is a program code obtained by correcting the processing for the program code A.

In FIG. 12, when a token is additionally inserted into a certain program code as described above, the clone pieces are discontinuous in a state shifted in the horizontal right direction or the vertical downward direction.
Accordingly, if a token is simply added to a certain program code, all discontinuous portions are shifted in a certain direction with respect to the horizontal right direction or the vertical downward direction. Therefore, if all the discontinuities are shifted horizontally right or vertically downward, the program code is simply a token added to the original program code. Program code with added tokens can be considered a child.

  If it is shifted horizontally to the right, the program code (template) indicated by the vertical axis of the clone analysis diagram becomes a parent, and the program code (template) indicated by the horizontal axis becomes a child to which a token is added. If it is shifted vertically downward, the program code (template) indicated by the horizontal axis of the clone analysis diagram becomes a parent, and the program code (template) indicated by the vertical axis becomes a child to which a token is added. In the example of FIG. 13, the discontinuous portion 1311 of the clone analysis diagrams 1301, 1302, and 1304 corresponds to these examples.

For example, in the clone analysis diagram 1301 in FIG. 13, it can be seen that since the program code A has only a token added to the template A, template A> program code A. “>” Indicates “parent> child”.
Similarly, in the clone analysis diagram 1302, it can be seen that template A> program code B and clone analysis diagram 1304 that program code A> program code B. Therefore, the system diagram creation unit 110 derives a parent-child relationship of template A> program code A> program code B from the clone analysis diagrams 1301, 1302, 1304.

Furthermore, as shown by reference numeral 1321 in the clone analysis diagram 1305, the portion where the token has been modified or replaced with a different token is in a state where the clone piece has been completely broken. This is referred to as a tearing portion.
When there is such a broken portion or a discontinuous portion in which the clone piece is shifted in an oblique direction as indicated by reference numeral 1322 in the clone analysis diagram 1305, the clone analysis diagram as shown in the clone analysis diagram 1301, 1302, 1304 The parent-child relationship cannot be derived from the shape.
In FIG. 13, clone analysis diagrams 1303, 1305, and 1306 correspond to such clone analysis diagrams.

  When the parent-child relationship cannot be derived from the shape of the clone analysis diagram as shown in the clone analysis diagrams 1303, 1305, and 1306 in FIG. Thus, it is determined that the structure having the longest average length of the clone code pieces between the program codes is close in structure, and the parent-child relationship is derived based on this derivation rate.

Derivation rate = (T ₁ + T ₂ +... + T _N ) / N (2)

In the formula (2), N is the number of clone code fragments, and T _n is the length of the nth clone code fragment.
The system diagram creation unit 110 calculates Equation (2) for each of the clone analysis diagram 1305 (program code A-program code D) and the clone analysis diagram 1306 (program code B-program code D) in FIG. Then, the average length (derivation rate) of clone code fragments is obtained. Since template A has a parent-child relationship with all program codes, system diagram creation unit 110 removes clone analysis diagram 1303 from the derivation rate calculation target.

  Assuming that the calculation result of the derivation rate is derivation rate of program code A−program code D> program code B−derivation rate of program code D, the system diagram creation unit 110 determines that program code D is program code from program code B. It is determined that the structure is close to A. Then, the system diagram creation unit 110 positions the position of the program code D in the system diagram in parallel with the program code A and sets it as a child of the template A that is the parent of the program code A. When the derivation rate of program code B-program code D> program code A-derivation rate of program code D, the system diagram creation unit 110 sets the position of the program code D in the system diagram in parallel with the program code B. The program code D is positioned as a child of the program code A which is the parent of the program code B.

As described above, when the procedure for deriving the parent-child relationship by the system diagram creation unit 110 is organized,
1. The system diagram creation unit 110 determines whether or not all the discontinuous parts in the clone analysis diagram are displaced in a certain direction with respect to the horizontal right direction or the vertical downward direction. The parent-child relationship is derived from the direction in which they are shifted (clone analysis diagrams 1301, 1302, 1304 in FIG. 13).
2. The system diagram creation unit 110 excludes the clone analysis diagram having the template 202 from the clone analysis diagram (clone analysis diagram 1303, 1305, 1306) from which the clone pieces are not shifted in a certain direction from the calculation target of the derivation rate (clone analysis diagram). 1303).
3. The derivation rate is calculated from the clone analysis chart that was not excluded from the derivation rate calculation target, and the parent-child relationship is derived.

Note that the system diagram creation unit 110 may determine the derivation relationship so that the program code created before becomes a parent by referring to the development order of the systems registered by the user.
In this way, the system diagram creation unit 110 obtains a derivation relationship between program codes, creates a system diagram, and displays the created system diagram on the result display unit 300.

(S114: System diagram creation process)
FIG. 14 is a diagram showing an example of a system diagram for explaining the system diagram creation processing in step S114.
Through the processing described with reference to FIG. 13, the system diagram creation unit 110 derives the relationship of the program code A and the program code D as children of the template A and the program code B as a child of the program code A (grandchild of the template A). A system diagram 1401 is displayed on the display unit 300.

"Concrete example"
Next, with reference to FIGS. 15 to 19, a procedure for creating a system diagram according to the present embodiment will be described using a specific example.
In this embodiment, the language used for the source program to be calculated for the system diagram is C language. Here, it is assumed that similar program codes are calculated from two analysis target program code groups α and β for a certain template 202, and a system diagram is calculated.
First, prior to the similarity calculation, the user registers the program code group α as an analysis target and the program code group β created based on the program code group α as an analysis target program code.

  Here, the program code group 201 is another program code group 201 configured using a common program code. For example, programs with different versions correspond to the program code groups α and β. The registered program code is subjected to lexical analysis in step S102 of FIG. 3, and further processed in steps S104 to S106 to create a template 202, a function dictionary 203, and a memory access dictionary 204.

FIG. 15 is a diagram illustrating specific examples of templates, function dictionaries, and memory access dictionaries.
In the example of FIG. 15, the “common function (...)” Of the template 202 corresponds to “sub_A (), sub_B ()” by the function dictionary 203, and the function memcpy refers to the memory access dictionary 204. It can be seen that “pData” is a global variable.
In this manner, meaningful tokens used in the analysis target program code group 201 are defined in combination. Here, a meaningful token is a token having a copy relationship with each other. For example, as shown in FIG. 15, a structure using a memset function, an if-else statement, a common function, and a memcpy function is defined as a template 202. In the template 202 of FIG. 15, it is assumed that the functions “common function” and “memcpy” have important meanings, the weights of “common function” and “memcpy” are “2”, and the weights of other tokens are “ 1 ”.

FIG. 16 is a diagram illustrating specific examples of templates and program codes.
The template a in FIG. 16 is created based on the program code a1 in FIG. Furthermore, the program code c is a program code created by making corrections based on the program code a1.
The number of tokens registered in the template a is 5, and the maximum score is 7 according to the weight in FIG.
In the program code a1, all tokens defined in the template a appear in the order registered in the template a (dot portion). If “& pRtn” of “memcpy” is a global variable, the similarity of the program code a1 to the template a is 100%. Whether a variable used in a function is a global variable or a local variable can be determined from the form of a declaration statement.
In the program code c, all tokens registered in the template a appear, but the order of appearance differs from the template a, such as “if” appearing after “Sub_B”. The “if-else” statement is not determined as a clone code of the template a, and only “memset”, “Sub_B”, and “memcpy” are determined to match the template a (dot portion). As a result, the similarity of the program code c to the template a is approximately 71%.

That means
Similarity of program code a1 = (1 + 1 + 2 + 2 + 1) / (1 + 1 + 2 + 2 + 1) × 100 = 100 (%)
Similarity of program code c = (1 + 2 + 2) / (1 + 1 + 2 + 2 + 1) × 100≈71 (%)
It is.

  As described above, the program analysis apparatus 1 calculates the similarity for each program code in the program code groups α and β.

FIG. 17 is a diagram illustrating a similarity result of each program code constituting the program code group α with respect to the template a.
Here, “a1”, “c”, “c1”, “d”, “e”, “e1”, “h”, and “j” are names of program codes constituting the program code group α.
Here, only a program code having a threshold of 40% and a similarity of 40% or more is displayed.

18 and 19 are diagrams showing system diagrams of the program code groups α and β. In FIG. 18 and FIG. 19, only “a” is the template 202, and the others are program codes.
These are the results of individually performing the processing shown in FIG. 3 on the program code group α and the program code group β.

Further, as shown in FIG. 19, the system diagram creation process may integrate the results of FIG. 18 and display them on the result display unit 300.
This may be created, for example, when the user inputs via the user interface unit 400 that the program code group β is created based on the program code group α.

  The present embodiment is not limited to the C language as long as it is a program system having a plurality of program codes, and can be used in other program languages.

<Summary>
According to the present embodiment, by using the function dictionary 203 that registers functions that can be regarded as the same function even if the formats are different, and the memory access dictionary 204 that registers the types of variables used in the functions, Are extracted by simple comparison of program code by assuming that the tokens are substantially the same even if they are different, or that they are the same in form, but different tokens (functions) if the referenced variable is different. Token types that cannot be extracted can be extracted, and highly accurate program analysis can be performed. Furthermore, by creating a system diagram in consideration of the token type, it is possible to derive a parent-child relationship between program codes with high accuracy.

DESCRIPTION OF SYMBOLS 1 Program analysis apparatus 100 Control part 101 Analysis object program code group registration part 102 Lexical analysis part 103 Template creation part 104 Function dictionary creation part 105 Memory access dictionary creation part 106 Similarity calculation part 107 Similarity analysis result creation part 108 Classification result creation Unit 109 Clone analysis unit 110 System diagram creation unit 200 Storage unit 201 Program code group 202 Template 203 Function dictionary (token dictionary)
204 Memory access dictionary 205 Lexical analysis result 206 Token registration determination table 300 Result display section (display section)
1101, 1201, 1301-1306 Clone analysis diagram 1401 System diagram

Claims (8)

A program by a program analysis apparatus for deriving and displaying a parent-child relationship having a modification source program code as a parent and a modification destination program code modified by adding or correcting a token among a plurality of program codes An analysis method,
The program analysis device includes:
A function dictionary that registers functions that can be regarded as the same function, although the description format is different.
A memory access dictionary that registers the types of variables used in the function;
Is stored in the storage unit,
The program analyzer is
(A1) Perform a lexical analysis to decompose a plurality of program codes to be analyzed into the tokens,
(A2) A template that is generated based on the result of the lexical analysis in the predetermined program code, is stored in the storage unit, and has a token registered in the predetermined program code, and each lexical in each program code As a result of comparison with the token registered in the template with reference to the analysis result and the function dictionary, functions that have different description formats but can be regarded as the same function in the function dictionary are determined as the same function. Perform processing for each program code,
(A3) Although the template, the result of the lexical analysis in each program code, and the memory access dictionary are referred to and compared with the function registered in the template, the functions have the same format. , A function using different types of variables is determined as a different function, and the process of narrowing down the same function is performed for each program code,
(A4) Tokens related to the functions determined to be different functions are set as different tokens, and for two program codes of the plurality of program codes, the vertical axis indicates the appearance position of one program code token, and the other The appearance position of the token of the program code is taken on the horizontal axis, and when the same token appears in the two program codes, the coordinate of the horizontal axis corresponding to the appearance position of the same token is x, and the appearance position of the same token A clone analysis for generating data plotted at a point (x, y), where y is the coordinate of the vertical axis corresponding to
(A5) Based on the result of the clone analysis, a parent-child relationship between the template and the program code is derived,
(A6) The processing of (a4) and (a5) is performed for each template / program code pair to be analyzed,
(A7) A program analysis method, wherein a system diagram in which the template and the program code from which the parent-child relationship is derived is connected by a line is displayed on a display unit. A line in which the plot in the data of the result of the clonal analysis is continuous is a clonal piece,
The program analysis device is
When all the clone pieces are discontinuous in a state shifted in a certain direction with respect to the horizontal right direction or the vertical downward direction,
When all the discontinuous parts are shifted horizontally to the right, the template and program code indicated by the vertical axis of the clone analysis result are used as a parent, and the program code indicated by the horizontal axis of the result of the clone analysis is used as a child. ,
When all the discontinuous parts are shifted vertically downward, the template and program code indicated by the horizontal axis of the clone analysis result are set as a parent, and the program code indicated by the vertical axis of the clone analysis result is set as a child. The program analysis method according to claim 1, wherein: A line in which the plot in the data of the result of the clonal analysis is continuous is a clonal piece,
The program analysis device is
Calculate the derivation rate based on the following formula (1),
Two program codes having similar derivation rates are positioned in parallel in the system diagram,
The program analysis method according to claim 1, wherein a parent of the program code positioned in parallel is a common parent.
Derivation rate = (T ₁ + T ₂ +... + T _N ) / N (1)
Here, N is the number of clone code fragments, and T _n is the length of the nth clone code fragment. The program analysis device includes:
A template that associates weights with tokens used in a plurality of program codes is further stored in the storage unit,
The program analyzer is
2. The program analysis method according to claim 1, wherein in the program code to be processed, the similarity is calculated by adding a weight of a token that matches the token registered in the template. . The program analysis device is
The program analysis method according to claim 4, wherein a set of program codes whose similarity is less than a predetermined value is excluded from the processing targets of (a2) to (a5). The program analysis method according to claim 1, wherein the types of variables are a global variable, a local variable, and a plurality of global variables that are a plurality of global variables.   A program analysis program for causing a computer to execute the program analysis method according to any one of claims 1 to 6. A program analysis device that derives and displays a parent-child relationship between a plurality of program codes, the parent of which is the program code of the modification source and the child of the program code of the modification destination that has been modified by adding or modifying a token. And
A control unit for processing information; and a storage unit for storing information;
In the storage unit,
A function dictionary that registers functions that can be regarded as the same function, although the description format is different.
A memory access dictionary that registers the types of variables used in the function;
Is remembered,
The control unit is
(A1) Perform a lexical analysis to decompose a plurality of program codes to be analyzed into the tokens,
(A2) A template that is generated based on the result of the lexical analysis in the predetermined program code, is stored in the storage unit, and has a token registered in the predetermined program code, and each lexical in each program code As a result of comparison with the token registered in the template with reference to the analysis result and the function dictionary, functions that have different description formats but can be regarded as the same function in the function dictionary are determined as the same function. Perform processing for each program code,
(A3) Although the template, the result of the lexical analysis in each program code, and the memory access dictionary are referred to and compared with the function registered in the template, the functions have the same format. , A function using different types of variables is determined as a different function, and the process of narrowing down the same function is performed for each program code,
(A4) Tokens related to the functions determined to be different functions are set as different tokens, and for two program codes of the plurality of program codes, the vertical axis indicates the appearance position of one program code token, and the other The appearance position of the token of the program code is taken on the horizontal axis, and when the same token appears in the two program codes, the coordinate of the horizontal axis corresponding to the appearance position of the same token is x, and the appearance position of the same token The coordinate of the vertical axis corresponding to is y, and the data plotted at the point (x, y) is generated,
(A5) Based on the generated data, a parent-child relationship between the template and the program code is derived,
(A6) The processing of (a4) and (a5) is performed for each template / program code pair to be analyzed,
(A7) A program analysis apparatus characterized in that a system diagram in which the template and the program code from which the parent-child relationship is derived is connected by a line is displayed on a display unit.

Images