JP2009099030A - Processing content determination device and processing content determination method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a processing content determination device and processing content determination method capable of more accurately determining the sameness or similarity about processing contents described by using a prescribed relevant rule. <P>SOLUTION: A description rule analysis part 112 analyzes first and second comparison target description data 103 and 104 that are subjected to similarity or not determination to, for example, a tree structure by using a relation rule common to the both data, a similarity operating part 114 operates similarity between both comparison targets with a block pruned as one example by a comparison degree setting part 107 as a target. An operation result is output as similar or not, or the degree of similarity and displayed by a similarity determination display part 108. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention is a process for determining the presence or absence of identity of processing contents assembled using a predetermined relational law or detecting the degree of similarity, as in the case of comparing sentences using a related law called grammar. The present invention relates to a content determination device and a processing content determination method.

  It is assumed that there is a work A written in a certain language and a work B written in another language. In this specification, a copyrighted work refers to a created source or document. For example, when determining whether or not it is plagiarism, the presence or absence of similarity between two works A and B to be determined becomes a problem. Accordingly, various techniques for detecting similarities between the copyrighted works A and B have been proposed as techniques related to the present invention (see, for example, Patent Document 1 and Patent Document 2).

  Among these, Patent Document 1 discloses a technique for extracting a similar source code fragment from a source code described in a predetermined program language. In this first related technology related to the present invention, the conventional method of extracting similar source code pieces by comparing the source code included in the source code group with the brute force is the same as that of the source code included in the source code group. The problem that enormous processing time is required when a large number exists is solved. That is, in this first related technique, similar source code pieces are extracted with reference to a designated source code piece, and all source codes are compared in a brute force manner to extract similar source code pieces. Compared with the conventional method, the processing result can be obtained in a short time.

  Next, the second related technique described in Patent Document 2 will be described. The second related technique is a technique for confirming how much change has been made when the source program is changed.

  FIG. 4 shows an outline of the configuration of the source program comparison information generation system according to the second related technique. In the source program comparison information generation system 400, the source program reading unit 401 reads the source program 402 before change and the source program 403 after change. The basic comparison information analysis unit 404 compares the source programs 402 and 403 for matching / mismatching in line units, and extracts changed lines and comparative lines. And it classify | categorizes into the change block which shows the change part comprised from a change line or a comparison line, and another unchanged block. In this basic comparison information analysis unit 404, the source program is compared for each line, and is divided into an unmodified block composed of one or a plurality of continuous lines and a modified block where the modification has been performed.

  The detailed comparison information analysis unit 405 creates detailed comparison information for a combination of a comparison line and a change line in a change block that exists as an area sandwiched between unchanged blocks. Specifically, the row similarity calculation unit 411 calculates the similarity of each row in the changed block for all combinations of rows for both the source program 402 before change and the source program 403 after change.

  Next, based on the row similarity for each combination calculated by the row similarity calculation unit 411, the block similarity calculation unit 413 calculates the similarity for the entire changed block for all the combination patterns in the changed block. Do.

  The change type determination unit 414 provided last determines the change type for each row of the change block based on the combination pattern having the maximum block similarity as a result of the above processing. Specifically, in the combination pattern, when the comparison line and the changed line are associated with each other, it is assumed that “correction” has been made for the line. In addition, regarding a comparison line that is not associated, it is assumed that the line is “deleted”, and when there is a modified line that is not associated, it is determined that the line is “added”. I am doing so.

  Next, an outline of techniques related to the present invention other than patent documents will be described. Explanations regarding CFG (Context Free Grammar) and BNF (Backus Naur Form) are provided as the third related technology of the present invention (see, for example, Non-Patent Document 1), and ANSI C (American National Standard for Information). Systems-Programming language C) (89) and BNF implementation examples in other languages are performed as the third related technique of the present invention (see, for example, Non-Patent Document 2 and Non-Patent Document 3). Further, a description of DCG (Definite Clause Grammar) by Prolog (PROgramming in LOGic) is described in Non-Patent Document 3 in the third related technology. However, “JISX3012: 2001 Prolog” includes only the basic part and does not include DCG. In the conventional technique related to the present invention, the source character string difference is extracted.

Although various techniques related to the present invention have been described above, these techniques for detecting similarities between works A and B generally have the following tendencies.
(A) Compare source codes and documents directly as character strings instead of their structures.
(B) When comparing character strings, the comparison is performed in a single language, and the comparison between different languages is not performed.
(C) No block is set as a comparison unit.

For example, in the second related technique of the present invention shown in FIG. 4, a changed part composed of a comparison line on the pre-change source program side corresponding to a changed line or a changed part on the changed source program side is used as a changed block. Blocks (areas) are classified in such a way that unchanged parts are regarded as unchanged blocks. In other words, in the second related technique, “block” is only for distinguishing between a changed portion and an unchanged portion, and the similarity is statistically obtained by taking a difference of these source programs mainly in line units. Measured by a typical method.

JP 2006-018693 A (paragraph 0022, FIG. 3) Japanese Patent Laid-Open No. 2003-280903 (paragraphs 0010 and 0011, FIGS. 1 and 3) Kenji Satsuki, “bison / flex program generator on MS-DOS” Keikaku Publishing, January 25, 1994, pp. 281-318 http://www.quut.com/c/ANSI-C-grammar-y.html http://www.csci.csusb.edu/dick/samples/index.html http://bach.istc.kobe-u.ac.jp/prolog/intro/lang.html

  As described above, when trying to determine the similarity of a work such as a source or a document, in the related technology of the present invention, the works A and B to be compared are made of a common language. The comparison was made by simply taking the difference. For this reason, the following problems have been pointed out.

(A) It was not possible to confirm the difference without being influenced by the degree of freedom of the source as a development tool. In other words, it is difficult to judge a large difference by ignoring a source difference that does not cause a difference between objects.
(B) Therefore, for example, when trying to determine whether there is infringement of patent rights or copyrights, it is as if each similar source is detected and this is simply replaced with the original source. It is important to determine the identity of the structure to make a determination, but it has been difficult to determine the identity of such a structure.
(C) As a result, it has been poor at detecting similar documents that infringe patent rights and copyrights.

  For example, in the second related technique, the difference of the source program is mainly taken in line units. Therefore, if this is applied to a technique for discriminating the similarity of a general work, it is possible to search for a place where the specified work is illegally tampered with, or two of the works A and B. This is a useful technique for applications such as searching for changes in works between points. However, we want to confirm the identity of the results of recognizing the difference in the structure of the copyrighted works A and B and visualizing the difference, and the result of porting the software to another language or another platform, independent of language, or There was a problem that it was not possible to respond to the need to confirm unauthorized diversion just by replacing text strings and languages.

  In the above, the determination or determination of the identity or similarity of the copyrighted work has been described. However, there are similar problems when performing general similarity determination or similarity determination based on a predetermined related law. .

  Accordingly, an object of the present invention is to provide a processing content determination apparatus and a processing content determination method capable of more accurately determining the identity or similarity of the processing content described using a predetermined relational law.

  In the present invention, (a) a comparison object description input means for inputting each description to be compared using a predetermined relational law and to be compared for similarity determination, and (b) the description to be compared is described above. A description composition contents relation map generating means for generating a description composition contents relation map representing each relation between the description composition contents constituting the respective description contents, and (c) this description composition contents relation The processing content determination apparatus includes similarity calculation means for comparing the description structure content related maps generated by the map generation means with the descriptions to be compared with each other and calculating the degree of matching of these maps as similarity. Let

  In the present invention, (a) a comparison object description input step for inputting each description to be compared with a predetermined judgment rule and (b) a description to be compared, A description composition content relation map generation step for generating a description composition content relation map that analyzes each using the above-mentioned relational rules and generates a description composition content relation map that represents the relationship between the description composition contents constituting each description content; A processing content determination method including a similarity calculation step that compares the description structure content related maps generated in the content related map generation step with the descriptions to be compared with each other and calculates the degree of matching of these maps as a similarity To provide.

  In this way, in the present invention, each description that is assembled using a predetermined relational law and is compared for similarity determination is input, and the contents of each description using the above-mentioned predetermined relational law before comparing them. A description configuration content relation map representing the relationship between the description configuration contents constituting the. The description configuration content relation map is, for example, a parse tree as a map showing a hierarchical relationship or a network type data structure as a map representing the relationship with each other by a network. The calculation of similarity is performed by comparing the description configuration content related maps. Thereby, for example, the similarity (similarity or degree of similarity) can be calculated while obtaining the identity or difference between parents or children in a parent-child relationship, and more accurate determination can be made than in the prior art.

  FIG. 1 shows a configuration of a processing content determination system using a processing content determination device according to an embodiment of the present invention. The processing content determination system 100 includes a first comparison target description data storage unit 101 and a second comparison target description data storage unit 102. The first comparison target description data storage unit 101 stores the first comparison target description data 103, and the second comparison target description data storage unit 102 compares the first comparison target description data 103 with the first comparison target description data 103. Second comparison target description data 104 to be stored is stored. The first and second comparison target description data 103 and 104 are typically literary works such as novels and programs.

  The processing content determination apparatus 105 includes a comparison target description data input unit 111 that inputs the first and second comparison target description data 103 and 104, and the comparison target description data 103 input from the comparison target description data input unit 111. , 104, a description rule analysis unit 112 that analyzes a related rule (hereinafter referred to as a description rule), and an analysis description rule 113 that is analyzed, and the first and second comparison target description data 103, 104. The similarity calculating unit 114 is provided for calculating the similarity. Both of the comparison target description data 103 and 104 can be considered as a set of blocks configured by description rules. Here, the description rule analysis unit 112 analyzes the description rule with reference to the external description rule database 106.

  That is, the description rule analysis unit 112 is a part that analyzes a description rule common to the first and second comparison target description data 103 and 104. For example, if this is described by a related law (in this case, a language) such as a novel, a patent specification, or a source program, preprocessing for unifying it is necessary if the language is different. As an example, if the first comparison target description data 103 is described in English and the second comparison target description data 104 is described in Japanese, these are expressed in a common language (English, Japanese or Esperanto). It is necessary to perform preprocessing for translation into a common language of both languages, such as words. The description rule database 106 may include a database portion for executing such a preprocessing portion.

  If the description contents are different, the preprocessing method is also different. For example, when the content of the description is not the language but the content of the game, when discussing the similarities of the copyrights of the salaryman advancement game and the politician advancement game, consider the various factors such as the role of the characters to be compared. The accuracy of similarity determination is increased by accurately performing preprocessing for rearranging both in the same game world or common game world.

  The description rule analysis unit 112 analyzes the contents of the one translated into the common language as described above or originally written in the common language. Therefore, the first comparison target description data storage unit 101 and the second comparison target description data storage unit 102 store the comparison target description data 103 and 104 created by using the related laws that can analyze the contents, respectively. Need to be. For example, a language such as Japanese has a related law called grammar, which can analyze the content. It is possible to apply the present invention as the first and second comparison target description data 103 and 104 for a description in which a relational law capable of analyzing the connection relation of each part is specified as in the game described above, not limited to language. Of course.

  Here, the analysis by the description rule analysis unit 112 refers to, for example, examining how each content is related in the first comparison target description data 103 using a description configuration content related map. For example, to investigate hierarchical relationships such as parents, children, grandchildren, etc., and relationships that are connected via networks such as people in the same department, people belonging to the same hobby club, and the same age. .

  The similarity calculation unit 114 compares the corresponding part or the corresponding group of the description rule 113 analyzed for the first and second comparison target description data 103 and 104, respectively, and whether or not the corresponding ones are identical. The amount of difference is measured, and the degree of similarity between the two descriptions is calculated as an analog result, or the binary result of similarity is calculated. At this time, the degree of comparison (depth or range) may be set by the comparison degree setting unit 107. For example, as long as the similarity of the description data having the hierarchical analysis result is compared, the similarity calculation may be performed from the basic hierarchy to a predetermined hierarchy. In the embodiment described below, this is realized by the concept of “pruning”.

  The calculation result 115 of the similarity calculation unit 114 is output to the similarity determination display unit 108. The similarity determination display unit 108 may output to a recording medium such as a printer, or may output display contents such as a liquid crystal display. Further, the processing content determination device 105 and the similarity determination display unit 108 may be connected by a communication cable or may be connected via a communication network. Further, a storage device such as a hard disk for temporarily or semi-permanently storing the calculation result 115 may be interposed between them.

  The calculation result output to the similarity determination display unit 108 may represent a degree of similarity as described above, or may be a determination of similarity. The output form varies depending on the purpose of use of the processing content determination apparatus 105. The comparison degree setting unit 107 is free to set such an output form.

  According to the processing content determination apparatus 105 of the present embodiment as described above, similarity determination is performed by performing preprocessing such as translation even between comparison target description data having different description formats. And the restriction of the comparison target is relaxed. In addition, since the similarity determination is performed based on the description rule, the natural phenomenon is not limited to the grammar. For example, the similarity determination of two typhoons is performed using a description about the weather assembled using a predetermined related law. It is possible to make a wide range of similar judgments such as

  Furthermore, since a similar determination is made based on the related laws, an objective determination is possible. Further, if the comparison level is set, efficient processing can be performed according to the processing purpose of the processing content determination apparatus 105.

  1 is realized as a computer using a CPU (Central Processing Unit) or processor (not shown) and a storage medium such as a hard disk storing a control program for similarity determination (not shown). be able to. In this case, the first comparison target description data storage unit 101, the second comparison target description data storage unit 102, and the description rule database 106 can be configured by a server arranged on a hard disk or a communication network, and a comparison degree setting is performed. The unit 107 can be configured by a keyboard or a pointing device (not shown).

  FIG. 2 shows an outline of the configuration of a processing content determination apparatus according to an embodiment of the present invention. The processing content determination apparatus 200 includes a meta language 203 that reads a meta language source 201 and outputs a corresponding syntax analyzer 202. Here, a meta language is a tool that generates a grammar analysis function of an existing language. Typically, “Yacc (Yet Another Compiler Compiler)” and “Prolog” can be cited as tools for automatically generating a C program for performing syntax analysis.

  The first source / language 204 and the second source / language 205 as the copyrighted work input to the processing content determination apparatus 200 are described in different languages. The first syntax analyzer 206 reads the first source / language 204 and outputs a first syntax analysis tree 207. The second parser 208 reads the second source / language 205 and outputs a second parse tree 209.

  The parse tree comparator 211 reads the similarity determination criterion 213 before the comparison in the same manner as the comparison unit criterion 212. Next, the parse tree comparator 211 reads the first parse tree 207 and the second parse tree 209 to be compared. Then, according to the condition specified by the comparison unit criterion 213, the first and second parse trees 207 and 209 are each divided into sets of subtrees. Next, the parse tree comparator 211 compares each subtree of the first parse tree 207 with the brute force combination of each subtree of the second parse tree 209. For all the comparison results, the similarity is determined based on the designation in the similarity determination criterion 213, and the comparison result list 214 is output as the comparison result.

  As described above, the processing content determination apparatus 200 according to the present embodiment includes a processing part that creates the formal language parser 202, a processing part that generates the parsing trees 207 and 209 for each language, and these parsing trees 207, 209 is compared. Of these, the process of generating the parse trees 207 and 209 for each language is performed only when necessary. The processing content determination apparatus 200 can take many configurations, but in the present embodiment, only a specific pattern is shown.

  Incidentally, the syntax analyzer 202 in the processing content determination apparatus 200 shown in FIG. The generated parser 202 is used as a first parser 206 and a second parser 208 as indicated by broken lines. In this way, instead of creating and using two different types of parsers 206 and 208, one type of parser may be used.

  FIG. 3 specifically shows the configuration of the parse tree comparator. The processing content determination apparatus 200 of this embodiment will be described with reference to FIGS. 2 and 3. In this embodiment, the meta language 203 is handled. Therefore, in the drawing, the data is not distinguished from the notation of these processing steps. This is because data and these processing steps can be partly described, but the processing target is also data, so if the same thing is changed for each part, it may look different. .

  First, in step S301 as the first processing procedure, the meta language 203 reads the meta language source 201 and outputs the corresponding parser 202 as described above. The meta language source 201 includes, for example, a language grammar describing a work to be compared and a side effect of converting the grammar structure into a tree structure.

  The language of a copyrighted work is a natural language as long as it can be interpreted as a formal language based on a theory capable of information processing, such as CFG (Context Free Grammar) and DCG (Definite Clause Grammar). Or an artificial language. However, in practice, a language that can be processed in the language such as “Yacc” (Yet Another Compiler-Compiler) or “Prolog” (PROgramming in LOGic) is required. “Prolog” was originally developed for natural language analysis.

  Further, as an ingenuity in implementation, the meta language source 201 is processed in advance in a language system that requires a preprocessor. The parser 202 output from the meta language 203 is used as the first and second parsers 206 and 208 in step S302.

  In the next step S302, the first and second syntax analyzers 206 and 208 obtained from the meta language 203 and the meta language source 201 read a work described in each target language. Here, the first source / language 204 and the second source / language 205 are read as works. The first syntax analyzer 206 reads the first source / language 204 and outputs a first syntax analysis tree 207. The second parser 208 reads the second source / language 208 and outputs the second parser tree 209. For example, the first parser 206 has a function of generating a parse tree from a C language source, and the second parser 208 has a function of generating a parse tree from a BASIC language source. Think of it as prepared.

As described above, the first and second parsing trees 207 and 209 are expressed in step S302, so that two works with different languages have an equivalent description format, and a formal comparison is possible. Of course, there are various output formats such as tree structure, network structure, XML (Extensible Markup Language), etc., but in order to compare the two, it is necessary to unify the expression format to one. . For example, a procedural language as a programming language that executes written instructions sequentially and changes the contents of variables according to the results of processing consists of ordering, conditional branching, repetition, and jumping. . This is replaced by the following first to fifth relational rules R _{1 to} R ₅ in XML. For example, for the network structure, it is necessary to examine the conversion format itself, but this is out of the scope of the present invention, and thus the description is omitted.

Related Law R ₁ ...... “Sequential” is a sequence of tags of the same level corresponding to each sentence.
Related rule R ₂ ...... “Conditional branch” is a sequence of tags at the same level as the branch conditional expression.
Related Law R ₃ ...... “Repetition” is a sequence of tags having the same level as the repetition conditional expression.
Related law R ₄ ... “Jump” is a tag that points to a node obtained by converting a token indicating a label.
Related Law R ₅ …… “Direct Recall” is not treated specially as a tag (comparison between tag options).

Here, the “direct recursive call” of the related rule R ₅ is to call the same procedure from among the procedures, and the original procedure (a) is further transferred from the procedure (eg, b) called from the procedure (eg, a). It is distinguished from indirect recursive calls such as calling. A token is the smallest unit of recognition without white space or comments.

Taking the C language as an example, an example in which the first and second parsing trees 207 and 209 are converted to XML according to BNF (Backus Naur Form) as one of meta languages will be given. However, in the example of the related rule R ₁ , the BNF is too complicated and the parallel enumeration of essentially equivalent sentences at the same depth is the related rule in the present embodiment, and is not essential. It is not illustrated here.

Example of related law R ₂
[Conditional branch BNF]
selection_statement
: IF '(' expression ')' statement-block
| IF '(' expression ')' statement-block ELSE statement-block
| SWITCH '(' expression ')' statement-block
;
[Conditional branch XML (IF '(' expression ')' statement-block ELSE statement-block only)]
<select key = 'if' expression = ''>
<statement-block eval = 'true'>… </ statement-block>
<statement-block eval = 'false'>… </ statement-block>
</ select>

Example of related law R ₃
[Repeated BNF]
iteration_statement
: WHILE '(' expression ')' statement-block
| DO statement WHILE '(' expression ')'';'
| FOR '(' expression_statement expression_statement ')' statement-block
| FOR '(' expression_statement expression_statement expression ')' statement-block
| FOR '(' declaration expression_statement ')' statement-block
| FOR '(' declaration expression_statement expression ')' statement-block
;
[Repeated XML (For '(' expression_statement expression_statement expression ')' statement-block only example)]
<iteration key = 'for' eval = 'expression_statement'>
<statement-block>… </ statement-block>
</ iteration>

Example of related law R ₄
[Jumping BNF]
jump_statement
: GOTO IDENTIFIER ';'
| CONTINUE ';'
| BREAK ';'
| RETURN ';'
| RETURN expression ';'
;
labeled_statement
: IDENTIFIER ':' statement-block
| CASE constant_expression ':' statement-block
| DEFAULT ':' statement-block
;
[Jumping XML (GOTO IDENTIFIER ';' and IDENTIFIER ':' statement-block only)]
<jump key = 'goto' eval = '<IDENTIFIER>'/>
:
<label key = '<IDENTIFIER>'/>

Example direct recursive calls itself of the relevant law R ₅ is naturally contained in the BNF of R1~R4, own BNF is not present.
[Direct Recall XML]
<block rootid = '<FUNCTIONAL>'>
:
<statement procid = '<FUNCTIONAL>'>
:
</ Block>

  Here, “FUNCTIONAL” corresponds to a function in C language, and the one that matches is a recursion. In other procedural languages, a procedure name (for example, “PROCEDURE”) may be considered.

Although the description of the example of the related rule R ₁ is omitted, a conversion image of an arbitrary sentence is represented by a name “statement” tag here.

In the example described above, the tag represents the tree-structured node shown in FIG. Options are used to indicate the description and characteristics of the source work. The conversion image to XML is provisional. Therefore, the “statement-block”, “select”, “iteration”, “jump”, and “label” options in the XML tag are words such as “key” corresponding to the reserved word and “eval” corresponding to the evaluation. It is not necessary to have the concept of In the XML tag, a structure such as the relation law R ₁ to the relation law R ₅ may be determined by the tag name, option, and tag hierarchy.

  In this example, “key” is an example that means a reserved word in an artificial language that matches the BNF expression. The uppercase character string indicates an arbitrary or specific token (reserved word or user-defined word) in the C language. In the example, “eval” represents a user-defined word or an expression.

  The language system of the formal language may be other than a structured procedural language such as C language. For example, in the case of a functional language or a logic language, the sentence itself has a tree structure, and can be expected to be used as it is for determining the similarity of the source processing methods. On the other hand, an unstructured procedural language may be difficult to represent in a tree structure.

  Some natural languages, such as English, are relatively easy to handle as formal languages. Such a natural language can be applied to similarity determination of documents. However, even in such natural language, some prose poems do not have a regulated grammar system, or some can be explained by modified generation grammar, but a parse tree is generated due to loss of punctuation, etc. It is thought that processing to do is difficult.

  Note that the processing content determination apparatus 200 according to the present embodiment does not necessarily pursue strictness regarding syntax analysis. In other words, if the syntax can be analyzed even at the summary level, a language-independent parse tree can be generated to determine the similarity of the works to be compared. For this reason, the application range of the present invention is wide. In the present invention, it is possible to include a token of a copyrighted work and its type as attributes of the parse tree.

Next, step S303 will be described.
The parse tree comparator 211 compares the first and second parse trees 207 and 209 in the following manner.

  The first parse tree comparator 211 first reads the comparison unit criterion 212 and the similarity determination criterion 213 before comparison. The reading order of the comparison unit criterion 212, the similarity determination criterion 213, and the first and second syntax analysis trees 207 and 209 may be arbitrary.

  Next, the parse tree comparator 211 divides the read first and second parse trees 207 and 209 into smaller unit blocks (branches). This division follows a block division rule (pruning rule). This block division rule will be described.

  The comparison unit reference 212 divided into blocks is implemented as an environment setting (setting file, registry, environment variable, etc.). A condition such as a block unit when the block is formed based on the comparison unit criterion 212 is set, and the parse tree comparator 211 reads the set condition before the comparison.

  The essential condition of the conditions for blocking is used to specify a unit to be divided from a root such as a function definition or a method definition. In the example of conversion to XML shown above, the depth of the tag hierarchy is specified, and pruning is performed with the depth greater than that as a block when the parse tree is read. As a specific unit to be blocked, a group (for example, a C language function) that is not a procedure part or a method level scope is assumed.

  In languages like C #, there is a class and namespace scope above the function scope. For this reason, there is not always a function directly under the root. Therefore, the block division rule needs to be adjusted. Further, as an optional incidental condition to the condition for blocking, branches that are too low as a result of blocking (0 hierarchy and 1 hierarchy) can be excluded from comparison targets. Depending on the purpose of the comparison, there will be a difference between important and unnecessary. For example, if you ask for the matching of the algorithm, it is meaningless. By omitting this, the number of combinations will be reduced and the processing time will be greatly shortened. Can do.

  On the other hand, if you ask for copyright infringement of a document, you will also need to compare at a lower branch level. The arbitrary accompanying conditions described above are provided for the purpose of not comparing trivial subtrees, and do not mean that the end of the subtree to be compared is not seen.

  The pruning rules used in this embodiment are top-down. For this reason, a subtree of a comparison unit is generated from detection such as the entire syntax. As a result, although a trivial subtree also appears, a desired processing unit can be reliably extracted as a subtree. Many trivial subtrees can be easily determined. Therefore, if the trivial subtree is discarded at the stage where it is determined that it is a trivial subtree, the processing efficiency is good.

  Now, if the first and second parsing trees 207 and 209 are divided into finer unit blocks (branches), the first and second parsing trees 207 and 209 are set of these unit blocks (branches). Can be considered. As a special case, the block may be considered as a whole. The comparison is based on the combination of the branch X of the first parse tree 207 and the branch Y of the second parse tree 209 for each set of blocks (branches) of the first and second parse trees 207 and 209. Done. Since one unit block (branch) on one side to be compared is applied to the other, assuming that the number of unit blocks (branches) on the other side is n, the time complexity required for the search for comparison is O (n ) If hashing or a network type data structure is appropriately used, the amount of time calculation can be reduced to O (1).

  The parse tree comparator 211 makes a determination according to the similarity determination criterion 213 for all combinations. As the determination result, the combination of both blocks and the corresponding determination result are output as the comparison result list 214. Normally, the comparison result list 214 is created for all judgments, but the comparison result list 214 may be created for some judgments in some cases.

The above operation of the parse tree comparator 211 is shown in detail in FIG. In the example shown in FIG. 3, two works A and B are divided into blocks (A ₁ , A ₂ ,...) And (B ₁ , B ₂ ,...), Respectively, and the similarity criterion is used. The determination result is output as a comparison result list 214.

The similarity determination criterion includes the degree of coincidence and the determination criterion for determining it, and is classified as follows, for example.
(A) Completely consistent language system: When the structure and attributes match (for example, for copyright infringement detection)
(B) Partially matched language system: When the structures match (for example, for patent infringement detection)
(C) Disagreement

  In the present embodiment described above, the following effects can be obtained.

  In this embodiment, it is configured to compare the subtrees of the parse tree. For this reason, it is possible to compare the grammatical structures of works written in formal languages. The formal language includes not only an artificial language but also a natural language with a grammatical arrangement, so the range of applications according to this embodiment is wide. It is unclear how much the comparison between the procedural language, the functional language, and the logical language paradigm is meaningful, but the judgment criterion is meaningful for the comparison within each language paradigm. Even if it is a natural language, if it is a formal language, it is possible to confirm whether the grammatical structure of the translation is correct, for example, without infringing copyright.

  Further, in this embodiment, similarity is compared using a parse tree, so that processing can be performed independently of the type of formal language. Further, since the subtree is used as the comparison unit, whether or not the trivial subtree is to be compared can be adjusted as necessary, and the comparison can be performed in an appropriate unit.

  <Modification of Example>

  Of course, the present invention is not limited to the embodiments described above. For example, in the previous embodiment, a hierarchical network using a tree structure of a parse tree is used. However, this parse tree may be expressed by a network type data structure. This will be described as a modification.

  A node in the network data structure is either a root as a node at the top, a branch point, or a leaf as a node without a node below. In this modification, the route is the initial node. Next, each token is numbered according to the order of appearance in the analysis. These tokens, which are the smallest unit of recognition without white space and comments, are treated as nodes.

  Corresponding to the XML example, if there is a block scope (corresponding to a statement-block) that appears during conditional branching and repetition, or in some cases, sequential processing, branching is performed there.

  Conditional branching and repetition are not different from each other as a different control flow as the technical idea of the present invention. For this reason, only one of the structures and the branching and repeating conditions are held as attributes of each node. However, in the case of processing performed on the network data structure in this modification, it should be noted that each subtree obtained by pruning must be re-recognized as a separate network. As a result, the unique numbers assigned to all nodes in each branch are reassigned in order with the same value for the route.

  According to this processing, the unique number of the branch connected to the node is also recalculated and reassigned to each branch. The reason why the unique numbers of nodes and branches (branches) are reassigned is because the network type data structure taken up depends on the assigned value. This is because it is determined blindly.

  Therefore, in the case of this modification, the parse tree comparator 211 shown in FIG. 2 can make a predetermined determination regarding similarity by comparing the unique numbers of nodes and branches with the attributes of each node. It becomes like this.

  The adoption of such a network type data structure can be expected as an alternative to XML with poor processing efficiency. When processing a network type data structure, it is effective to replace it with a one-dimensional data structure that can be easily handled by a computer. In this regard, for example, a technique disclosed in a related technique (Japanese Patent Laid-Open No. 10-078911) proposed by the present inventor can be used.

  The embodiments of the present invention described above can also be applied to uses such as listing differences in structural units in the revision of a literary work. Conventionally, there has been a problem that confirmation is troublesome depending on the nature of the copyrighted work because of the difference confirmation in line units including the difference, but this can be solved. It can also be applied to applications such as checking grammatical structure in translation and listing structural identity that does not depend on character strings of works illegally diverted.

1 is a system configuration diagram of a processing content determination system using a processing content determination device according to an embodiment of the present invention. 1 is a configuration explanatory diagram of a processing content determination apparatus according to an embodiment of the present invention. FIG. It is explanatory drawing which represented the structure of the parse tree comparator in a present Example concretely. It is a system block diagram showing the outline | summary of the structure of the source program comparison information generation system by a 2nd related technique.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Processing content determination system 101 1st comparison object description data storage part 102 2nd comparison object description data storage part 103 1st comparison object description data 104 2nd comparison object description data 105,200 Process content determination apparatus 106 Description Rule database 107 Comparison degree setting unit 108 Similarity determination display unit 111 Comparison target description data input unit 112 Description rule analysis unit 114 Similarity calculation unit 115 Operation result 201 Meta language source 202 Syntax analyzer 203 Meta language 204 First source language 205 Second Source / Language 206 First Parser 207 First Parser Tree 208 Second Parser 209 Second Parser Tree 211 Parse Tree Comparator 212 Comparison Unit Criteria 213 Similarity Criteria 214 Comparison result list

Claims (11)

Comparison target description input means for inputting each description to be compared for similarity judgment, which is assembled using a predetermined relational law,
Description structure content relation map generating means for analyzing the descriptions to be compared with each other using the relational law and generating a description composition content relation map representing the relation between the description composition contents constituting each description content; ,
The description composition content related map generated by the description composition content related map generating means is compared with the descriptions to be compared, and the similarity calculation means for calculating the degree of matching of these maps as the similarity. The processing content determination apparatus characterized by the above.   Each description to be compared using the predetermined relational law and to be compared in the similarity determination is a description having a grammatical structure, and the description configuration content related map generation means sets each description to be compared as a single description. Parse tree generating means for generating a language-independent parse tree by analyzing the grammatical structure of the grammar structure, and the similarity calculating means is a parse analysis of descriptions to be compared generated by the parse tree generating means. 2. The processing content determination apparatus according to claim 1, wherein the processing content determination device is a means for comparing trees and calculating a degree of coincidence of the contents of these structures as a similarity.   Each description to be compared using the predetermined relational law and to be compared in the similarity determination is a description having a grammatical structure, and the description configuration content related map generation means sets each description to be compared as a single description. Is a network type data structure generating means for generating a language-independent network, and the similarity calculating means is a network type of descriptions to be compared generated by the network type data structure generating means. 2. The processing content determination apparatus according to claim 1, wherein the processing content determination device is means for comparing data structures and calculating a degree of coincidence of the contents of these structures as a similarity.   The description structure content relation map generation means comprises pre-processing means for translating these into a common language when each description to be compared in the similarity determination is a different language. The processing content determination apparatus according to claim 3.   3. The process according to claim 2, wherein the similarity calculation means comprises block designation means for designating a description range to be compared as a block as a comparison range determined by a hierarchy depth in a tree structure. Content determination device.   In the syntax analysis of the parse tree generation means, a CFG (Context Free Grammar) analysis tool is used for a procedural artificial language, and a DCG (Definite Clause Grammar: definite) is used for a natural language. 3. The processing content determination apparatus according to claim 2, wherein a meta language having a clause grammar analysis function is used.   2. The processing content determination apparatus according to claim 1, wherein the similarity calculation result of the similarity calculation means includes a match or mismatch of the description to be compared and an intermediate aspect thereof.   Dividing means for dividing the parse tree generated by the parse tree generating means into a set of subtrees is provided, and the similarity calculating means compares the brilliant combinations of the subtrees and compares the similarity to the block. The processing content determination apparatus according to claim 2, wherein: A comparison object description input step for inputting each description to be compared with the similarity judgment, which is assembled using a predetermined relational law;
A description composition content relation map generation step for analyzing the descriptions to be compared with each other by using the relational law and generating a description composition content relation map representing a relation between the description composition contents constituting each description content; ,
The description composition content relation map generated in the description composition content relation map generation step is compared with the descriptions to be compared, and a similarity calculation step is performed to calculate the degree of matching of these maps as a similarity. The processing content determination method characterized by the above.   Each description that is constructed using the predetermined relational law and is a comparison target of similarity determination is a description having a grammatical structure, and the description configuration content relation map generation step includes a single description to be compared. Is a parse tree generation step for generating a language-independent parse tree by analyzing the grammatical structure, and the similarity calculation step is a parse analysis between descriptions to be compared generated by the parse tree generation step. The processing content determination method according to claim 9, which is a step of comparing trees and calculating a degree of coincidence of the contents of these structures as a similarity.   Each description that is constructed using the predetermined relational law and is a comparison target of similarity determination is a description having a grammatical structure, and the description configuration content relation map generation step includes a single description to be compared. A network-type data structure generation step for generating a language-independent network by analyzing the grammatical structure, and the similarity calculation step is a network type of descriptions to be compared generated by the network-type data structure generation step. 10. The processing content determination method according to claim 9, which is a step of comparing data structures and calculating a degree of coincidence of the contents of these structures as a similarity.

Images