JP6379666B2 - Document analysis apparatus, document analysis program, and document analysis method - Google Patents

Description

  The present invention relates to a document analysis apparatus, a document analysis program, and a document analysis method.

  Conventionally, when a language such as Chinese that does not have a word form conversion by part of speech is machine-translated, a noun phrase cannot be recognized as one unit due to the ambiguity of the part of speech, and there is a case where an incorrect translation is performed. On the other hand, recently, a technique for performing syntax analysis using an analyzed corpus database at the time of machine translation is known (for example, see Patent Document 1).

  In Patent Document 1, a sentence is cut into words based on grammatical rules and statistical methods and broken down into morphemes, and the sentence structure is analyzed based on the morphemes. As a result of this analysis, when there are a plurality of candidates that can be a correct syntax analysis result, a correct syntax analysis result is determined from the plurality of candidates based on the similarity to the corpus included in the analyzed corpus database.

JP 2007-241664 A

  However, since the sentence to be translated has various structures, there is a case where a correct parsing result cannot be obtained even if the parsing is performed based on the similarity with the corpus as described above.

  In one aspect, an object of the present invention is to provide a document analysis apparatus, a document analysis program, and a document analysis method that can accurately analyze a range of noun phrases in a sentence.

In one aspect, the document analysis apparatus includes a hypothesis unit that assumes a word group that appears multiple times in a document as a noun phrase candidate, and a plurality of correct syntaxes included in a syntax list, An extraction unit that extracts a syntax similar to a sentence, and whether the noun phrase candidate assumption is correct based on whether the extraction unit has extracted a syntax similar to a plurality of sentences including the noun phrase candidate. And a determination unit for determining whether or not.

In one aspect, the document analysis program assumes a group of words appearing multiple times in a document as a noun phrase candidate, and is similar to a plurality of sentences including the noun phrase candidate among a plurality of correct syntaxes included in a syntax list. To determine whether or not the assumption of the noun phrase candidate is correct based on whether or not syntax similar to a plurality of sentences including the noun phrase candidate can be extracted in the extracting process . A document analysis program that causes a computer to execute processing.

In one aspect, in the document analysis method, the assumption unit assumes a group of words appearing multiple times in the document as a noun phrase candidate, and the extraction unit includes the noun phrase among a plurality of correct syntaxes included in the syntax list. Each of the noun phrase candidates is extracted based on whether or not the syntax similar to a plurality of sentences including candidates is extracted, and the determination unit can extract the syntax similar to the plurality of sentences including the noun phrase candidates. This is a document analysis method for executing processing to determine whether or not the assumption is correct.

  The range of noun phrases in sentences can be analyzed with high accuracy.

It is a figure which shows roughly the hardware constitutions of the translation terminal which concerns on 1st Embodiment. It is a functional block diagram of a translation terminal. It is a figure which shows an example of a corpus after parsing. It is a flowchart which shows the process of the analysis part in 1st Embodiment. It is a figure which shows an example of the document of analysis object. It is a flowchart which shows the process of FIG.4 S12. Fig.7 (a)-FIG.7 (c) are figures which show an example of the data temporarily preserve | saved at a noun phrase candidate temporary storage part. FIG. 8A to FIG. 8D are diagrams for explaining a processing example of the noun phrase candidate character string of the noun phrase candidate ID (CANDI-1). It is a flowchart which shows the process of step S18 of FIG. FIG. 10A to FIG. 10C are diagrams (No. 1) for explaining a processing example of the noun phrase candidate character string of the noun phrase candidate ID (CANDI-2). FIGS. 11A to 11C are diagrams (No. 2) for explaining a processing example of the noun phrase candidate character string of the noun phrase candidate ID (CANDI-2). It is a flowchart which shows the process of step S20 of FIG. FIG. 13A and FIG. 13B are diagrams showing examples of noun phrase lists. FIG. 14A and FIG. 14B are diagrams for explaining a processing example of a noun phrase candidate character string of a noun phrase candidate ID (CANDI-3). It is a figure which shows the state which replaced the noun phrase of the document with the semantic symbol. It is a functional block diagram of the translation terminal which concerns on 2nd Embodiment. It is a flowchart which shows the process of the analysis part in 2nd Embodiment. It is a figure which shows an example of the document of the analysis object in 2nd Embodiment. FIGS. 19A to 19D are diagrams for explaining steps S12 and S13 'of FIG. It is a flowchart which shows the process of step S13 'of FIG. FIGS. 21A and 21B are views (part 1) for explaining step S20 'in FIG. FIGS. 22A to 22D are diagrams (part 2) for explaining step S20 'in FIG. FIG. 23A to FIG. 23D are views (No. 3) for explaining step S20 ′ in FIG. 17. It is a figure which shows an example of the noun phrase list corresponding to step S24 of FIG. It is a figure which shows a modification.

<< First Embodiment >>
Hereinafter, a first embodiment of a translation terminal as a document analysis apparatus will be described in detail with reference to FIGS. The translation terminal 10 according to the first embodiment is a terminal that translates a Chinese document into Japanese (translates between Chinese and Japanese).

  FIG. 1 shows a hardware configuration of the translation terminal 10. The translation terminal 10 is a terminal such as a PC (Personal Computer), for example, and as shown in FIG. 1, a CPU (Central Processing Unit) 90, a ROM (Read Only Memory) 92, a RAM (Random Access Memory) 94, and a storage A unit (here, HDD (Hard Disk Drive)) 96, a network interface 97, a display unit 93, an input unit 95, a portable storage medium drive 99, and the like. Each component of the translation terminal 10 is connected to a bus 98. The display unit 93 includes a liquid crystal display or the like, and the input unit 95 includes a keyboard, a mouse, or the like. In translation terminal 10, CPU 90 stores a program (including a document analysis program) stored in ROM 92 or HDD 96 or a program (including a document analysis program) read from portable storage medium 91 by portable storage medium drive 99. By executing the function, the function shown in FIG. 2 is realized. FIG. 2 also shows a parsed corpus 40 as a syntax list stored in the HDD 96 of the translation terminal 10.

  FIG. 2 shows a functional block diagram of the translation terminal 10. In the translation terminal 10, the functions of the analysis unit 50 and the translation unit 52 are realized by the CPU 90 executing the program, as shown in FIG. The analysis unit 50 performs syntax analysis of each sentence included in the input Chinese document to be translated (for example, a patent document as illustrated in FIG. 5), and outputs the syntax analysis result to the translation unit 52. The translation unit 52 translates each sentence into Japanese based on the syntax analysis result of each sentence.

  The analysis unit 50 includes an input reception unit 20, a noun phrase candidate extraction unit 22 as a hypothesis unit, a noun phrase replacement unit 24, a noun phrase candidate temporary storage unit 26, a consistency calculation unit 29, a syntax analysis unit 30, and an output unit 34. including.

  The input receiving unit 20 receives an input of a Chinese document to be translated, that is, a syntax analysis target, and transmits it to the noun phrase candidate extracting unit 22. The noun phrase candidate extraction unit 22 extracts a word group including a plurality of words appearing multiple times in the document as a noun phrase candidate character string. The word group detected as a noun phrase candidate character string is temporarily stored in the noun phrase candidate temporary storage unit 26. Here, as an example, the noun phrase candidate temporary storage unit 26 temporarily stores data as shown in FIGS. 7A to 7C.

  The noun phrase replacement unit 24 replaces the noun phrase candidate character string with a temporary noun ([**]) represented by a temporary meaning symbol based on the data temporarily stored in the noun phrase candidate temporary storage unit 26. The provisional noun [**] is a noun representing an arbitrary meaning.

  The consistency calculating unit 29 extracts, from among a plurality of correct syntaxes included in the parsed corpus 40, syntaxes similar to a plurality of sentences including noun phrase candidate character strings. The consistency calculation unit 29 determines whether the noun phrase candidate character string is a correct noun phrase based on the extraction result. Here, the parsed corpus 40 stores a plurality of examples of correct (parsed) syntax, and has a data structure as shown in FIG. Specifically, the parsed corpus 40 has fields of “example ID” and “syntax structure in which noun phrases are replaced with semantic symbols”. The field of “Syntax structure in which noun phrases are replaced with semantic symbols” stores the tree structure of the parsed syntax by expressions (A to C are words) such as ((A) (B)) (C). ing. Note that the expressions [method] and [soil] included in the syntax structure of the case ID = C1001 are symbols (also referred to as semantic symbols) that collectively represent words (synonyms / synonyms) that mean the method and soil.

  The syntax analysis unit 30 performs syntax analysis based on the determination result of the consistency calculation unit 29. The syntax analysis result is transmitted to the translation unit 52 via the output unit 34.

  Next, processing executed in the translation terminal 10 of the first embodiment will be described in detail along the flowcharts of FIGS. 4, 6, 9, and 12 with reference to other drawings.

  FIG. 4 is a flowchart showing an outline of processing executed in the analysis unit 50 of the translation terminal 10.

  In the process of FIG. 4, first, in step S10, the input receiving unit 20 waits until a Chinese document is input. When a Chinese document is input, the process proceeds to step S12, and the noun phrase candidate extraction unit 22 executes a subroutine of noun phrase candidate character string extraction processing. In the first embodiment, it is assumed that a patent document as shown in FIG. In FIG. 5, for convenience of explanation, each sentence included in the patent document is shown with an original sentence ID (Q301, Q302...).

  In step S12, processing according to the flowchart shown in FIG. 6 is executed. In the process of FIG. 6, in step S40, the noun phrase candidate extraction unit 22 extracts the text in the document in sentence units. In this case, a sentence is extracted for each original sentence ID shown in FIG.

  Next, in step S42, the noun phrase candidate extraction unit 22 extracts sentence pairs. For example, the noun phrase candidate extraction unit 22 extracts sentence pairs with original sentence ID = Q301 and Q302.

  Next, in step S44, the noun phrase candidate extraction unit 22 detects a partially matched character string of the sentence pair. In this case, the sentence pairs extracted in step S42 are compared, and a common character string portion is detected.

  Next, in step S46, the noun phrase candidate extraction unit 22 determines whether or not the partially matched character string is already detected when the partially matched character string is detected. If the determination is negative, the process proceeds to step S48, and the noun phrase candidate extraction unit 22 stores the noun phrase candidate temporary storage unit 26. In this case, in the noun phrase candidate temporary storage unit 26, for example, data having a structure as shown in FIGS. 7A to 7C is stored. The data of FIG. 7A to FIG. 7C includes items of “noun phrase candidate ID”, “noun phrase candidate character string”, and “appearing original sentence ID”. The item of “noun phrase candidate ID” stores identification information of noun phrase candidates such as CANDI-1, CANDI-2,. In the item “noun phrase candidate character string”, the partially matched character string detected in step S44 is stored. In the item “appearing original sentence ID”, the original sentence ID of the sentence in which the partially matched character string is detected is stored.

  After step S48, the noun phrase candidate extraction unit 22 proceeds to step S50. On the other hand, if the determination in step S46 is affirmative, the noun phrase candidate extraction unit 22 determines that the original text ID of the sentence pair extracted in step S42 is the corresponding data of the noun phrase candidate temporary storage unit 26 in step S49. Stored in the “appearance original text ID” field. Thereafter, the process proceeds to step S50.

  In step S50, the noun phrase candidate extraction unit 22 determines whether all sentence pairs have been extracted. If the determination here is negative, the noun phrase candidate extraction unit 22 returns to step S42 and repeats the processes and determinations of steps S42 to S50 described above. Note that, for example, when a sentence pair of the original sentence ID = Q302 and Q304 is extracted as a sentence pair, partial matching as shown as noun phrase candidate character strings in FIGS. 7 (a) to 7 (c). A character string is detected and stored in the noun phrase candidate temporary storage unit 26. Also, for example, when a sentence pair with the original sentence ID = Q304 and Q305 is extracted as a sentence pair, partial matching as shown as noun phrase candidate character strings in FIGS. 7B and 7C A string is detected. Further, for example, when a sentence pair with original sentence ID = Q306 and Q307 is extracted as a sentence pair, a partially matched character string as shown as a noun phrase candidate character string in FIG. 7C is detected. .

  Thereafter, all sentence pairs are extracted, and if the determination in step S50 is affirmed, the process proceeds to step S52. In step S52, the noun phrase candidate extraction unit 22 rearranges (sorts) the data stored in the noun phrase candidate temporary storage unit 26 in descending order of the number of characters in the noun phrase candidate character string. When the process in FIG. 6 is completed as described above, the process proceeds to step S14 in FIG.

  In step S14 in FIG. 4, the noun phrase replacement unit 24 identifies one noun phrase candidate character string. In this case, the noun phrase replacement unit 24 identifies data having the largest number of characters in the candidate character string among the data temporarily stored in the noun phrase candidate temporary storage unit 26. Here, it is assumed that the data shown in FIG. 8A (noun phrase candidate character string of noun phrase candidate ID = CANDI-1) is specified.

  Next, in step S16, the noun phrase replacement unit 24 replaces the noun phrase candidate character string in the document with the provisional noun [**]. Specifically, as shown in FIG. 8B, the noun phrase replacement unit 24 converts the sentence including the noun phrase candidate character string specified in step S14 from the document into the data in FIG. 8A. Are extracted based on the original text ID (= Q302, Q304). Then, as shown in FIG. 8C, the noun phrase replacement unit 24 replaces the portion of the noun phrase candidate character string in the extracted sentence with a provisional noun (provisional semantic symbol) [**].

  Next, in step S18, the consistency calculation unit 29 executes a subroutine of processing for extracting semantic symbol candidates for the noun phrase candidate character string. In the process of step S18, the process according to the flowchart of FIG. 9 is executed.

  In the process of FIG. 9, first, in step S60, the consistency calculation unit 29 acquires a sentence set Z (sentence number kmax) obtained by replacing a noun phrase candidate character string with a provisional noun. Here, it is assumed that the consistency calculation unit 29 has acquired two sentences (sentence number kmax = 2) shown in FIG.

  Next, in step S62, the consistency calculating unit 29 sets a parameter k indicating the number of processed sentences to 1. Next, in step S64, the consistency calculation unit 29 performs syntax analysis on the sentence zk (= z1) included in the sentence set Z, and obtains a syntax analysis result. Here, for example, it is assumed that the syntax analysis of the sentence with the original sentence ID = Q302 in FIG. 8C is performed and the result of the syntax analysis shown in FIG.

  Next, in step S66, the consistency calculating unit 29 searches the parsed corpus 40 for a parsing case similar to the parsing result. Note that it is assumed that there is no similar syntax analysis example for the original text ID = Q302 in FIG.

  Next, in step S68, the consistency calculation unit 29 determines whether there is a similar syntax analysis example as a result of step S66. If the determination is negative, the process proceeds to step S74, and the consistency calculation unit 29 determines whether k is kmax (= 2). If the determination here is negative, the process proceeds to step S76, and the consistency calculating unit 29 increments k by 1 (k ← k + 1), and returns to step S64.

  Returning to step S64, the consistency calculating unit 29 performs syntax analysis of the original sentence ID = Q304 in FIG. 8C as the next sentence z2, and obtains a syntax analysis result. Here, it is assumed that the syntax analysis result of the original sentence ID = Q304 in FIG. Next, in step S66, the consistency calculation unit 29 searches for a parsing case similar to the parsing result in the parsed corpus 40. Here, it is assumed that there is no similar parsing example. Therefore, the determination at the next step S68 is denied, and the process proceeds to step S74. In step S74, the consistency calculation unit 29 determines whether k is kmax (= 2). If the determination here is affirmed, the entire process of FIG. 9 is terminated, and the process proceeds to step S19 of FIG.

  When the process proceeds to step S19 in FIG. 4, the consistency calculation unit 29 determines whether or not a semantic symbol candidate has been extracted in the process of step S18. That is, the consistency calculation unit 29 determines whether or not the process of step S72 has been executed in the process of step S18 performed immediately before. If the determination in step S19 is negative, the process proceeds to step S26. The case where the determination in step S19 is negative means that the part extracted as the noun phrase candidate character string is not a correct noun phrase.

  If transfering it to step S26, the consistency calculation part 29 will judge whether all the noun phrase candidates were specified. If the determination in step S26 is negative, the process returns to step S14, and the noun phrase replacement unit 24 specifies a noun phrase candidate character string having the second largest number of characters. Here, as shown in FIG. 10A, it is assumed that a noun phrase candidate character string of noun phrase candidate ID = CANDI-2 is specified.

  Next, in step S16, the noun phrase replacement unit 24 replaces the noun phrase candidate character string in the document with the provisional noun [**]. Specifically, as shown in FIG. 10B, the noun phrase replacement unit 24 converts a sentence including a noun phrase candidate character string from the document into the appearance original sentence ID ( = Q302, Q304, Q305, Q306). Then, the noun phrase replacement unit 24 replaces the part of the noun phrase candidate character string with the provisional noun [**] in the extracted sentence.

  Next, in step S18, the consistency calculation unit 29 executes a process of extracting semantic symbol candidates for the noun phrase candidate character string (FIG. 9).

  In the process of FIG. 9, in step S60, the consistency calculation unit 29 acquires a sentence set Z (sentence number kmax = 4) obtained by replacing a noun phrase candidate character string with a provisional noun.

  Next, in step S62, the consistency calculating unit 29 sets a parameter k representing a sentence to 1. Next, in step S64, the consistency calculation unit 29 performs syntax analysis on the sentence zk (= z1) included in the sentence set Z, and obtains a syntax analysis result. Here, for example, it is assumed that syntax analysis is performed on the underlined portion of the original text ID = Q302 in FIG. 10B replaced with a temporary noun, and the syntax analysis result shown in FIG. 10C is obtained.

  Next, in step S66, the consistency calculating unit 29 searches the parsed corpus 40 for a parsing case similar to the parsing result. For the original text ID = Q302 in FIG. 10C, it is assumed that two cases ID = C1001 and C1002 shown in FIG. 3 are searched as similar syntax analysis cases.

  Next, in step S68, the consistency calculation unit 29 determines whether there is a similar syntax analysis example as a result of step S66. If the determination here is affirmed, the process proceeds to step S72, and the consistency calculation unit 29 identifies a semantic symbol candidate of the noun phrase candidate character string from the searched parsing case and stores it in the semantic candidate list. . In this case, as shown in FIG. 11A, in the searched parsing case (C1001), the semantic symbol corresponding to the provisional noun [**] of the sentence is [core]. In the syntactic analysis example (C1002), the semantic symbol corresponding to the provisional noun [**] of the sentence is [bacteria], so the consistency calculator 29 identifies these semantic symbols as semantic symbol candidates, Save to the semantic candidate list.

  Next, in step S74, the consistency calculation unit 29 determines whether k is kmax (= 4). If the determination here is negative, the process proceeds to step S76, and the consistency calculating unit 29 increments k by 1 (k ← k + 1), and returns to step S64.

  Returning to step S64, the consistency calculation unit 29 parses the sentence z2 in which the sentence with the original sentence ID = Q304 in FIG. To win. Here, it is assumed that the syntax analysis result of the original sentence ID = Q304 in FIG. Next, in step S66, the consistency calculation unit 29 searches for a parsing case similar to the parsing result in the parsed corpus 40. Here, it is assumed that a case of case ID = C1004 is searched as a similar syntax analysis case. In this case, the determination in step S68 is affirmed, and in step S72, the consistency calculation unit 29 identifies the semantic symbol [bacteria] as a semantic symbol candidate and stores it in the semantic candidate list. Thereafter, when the determination in step S74 is negative, the process proceeds to step S76, and the consistency calculating unit 29 increments k by 1 (k ← k + 1), and returns to step S64.

  Thereafter, the processes and determinations in steps S64 to S76 are repeated, and the consistency calculation unit 29 also performs syntax analysis on the remaining two sentences in FIG. In the first embodiment, for the sentence with the original sentence ID = Q305, the semantic symbol [bacteria] is identified as a semantic symbol candidate, and for the sentence with the original sentence ID = Q306, no semantic symbol candidate is identified. And

  Thereafter, when the determination in step S74 is affirmed, the process proceeds to step S19 in FIG.

  If transfering it to step S19, the consistency calculation part 29 will judge whether the semantic symbol candidate was able to be extracted in the process of step S18. That is, the consistency calculation unit 29 determines whether or not the process of step S72 has been executed in the process of step S18 performed immediately before. If the determination in step S19 is affirmed, the consistency calculating unit 29 proceeds to step S20.

  In step S20, the consistency calculation unit 29 executes a subroutine for consistency evaluation when the noun phrase candidate character string is a semantic symbol candidate. In step S20, specifically, processing according to the flowchart of FIG. 12 is executed.

  In the process of FIG. 12, first, in step S80, the consistency calculation unit 29 sets the parameter k indicating the number of sentence processes to 1, the parameter c indicating the specific number of semantic symbol candidates, and the parameter n used for consistency evaluation. Is set to 0. Here, the maximum value kmax of the parameter k is 4, and the maximum value cmax of the parameter c is 2.

  Next, in step S82, the consistency calculation unit 29 specifies a semantic symbol candidate Tc (= T1). Here, it is assumed that [core] is specified as the semantic symbol candidate Tc. Next, the consistency calculation unit 29 determines whether or not there is a semantic symbol candidate Tc (= T1) in the semantic candidate list of the sentence zk (= z1). As shown in FIG. 11B, the sentence with the original sentence ID = Q302 has a matching case (C1001) when the semantic symbol candidate Tc is [core], so the determination in step S84 is affirmed, Control goes to step S86.

  When the process proceeds to step S86, the consistency calculating unit 29 increments n by 1 (n ← n + 1), and the process proceeds to step S88. In step S88, the consistency calculating unit 29 determines whether k is the maximum value (kmax). If the determination is negative, in step S90, k is incremented by 1 (k ← k + 1), Return to S82. Thereafter, the consistency calculation unit 29 determines whether or not there is a semantic symbol candidate T1 in the semantic candidate list of all sentences, and proceeds to step S92 when k reaches the maximum value (kmax). .

If transfering it to step S92, the consistency calculation part 29 will calculate the consistency evaluation value in case provisional noun [**] is the semantic symbol candidate Tc (T1). Specifically, a consistency evaluation value is calculated from the following equation (1).
Consistency evaluation value = n / kmax (1)

  In the case of FIG. 11B, since n = 1, the consistency evaluation value is 1/4 = 0.25.

  Next, in step S94, the consistency calculation unit 29 determines whether c is the maximum value of c (cmax = 2). If the determination here is negative, the process proceeds to step S96, and the consistency calculating unit 29 returns k to 1, increments c by 1 (c ← c + 1), and returns to step S82.

  After returning to step S82, the processing / determination of steps S82 to S92 is executed as described above. Here, when [bacteria] is extracted as the semantic symbol candidate T2, the number n of sentences having the semantic symbol candidate T2 in the semantic candidate list is 3, as shown in FIG. Therefore, in step S92, the consistency calculation unit 29 calculates 3/4 = 0.75 as the consistency evaluation value.

  Thereafter, when the determination in step S94 is affirmed, the consistency calculation unit 29 ends all the processes in FIG. 12 and proceeds to step S22 in FIG. In step S22, the consistency calculation unit 29 determines whether there is a semantic symbol candidate having a maximum consistency evaluation value and exceeding a threshold value. For example, assuming that the threshold value is 0.5, in the example of FIG. 11B and FIG. 11C, the maximum value 0.75 of the consistency evaluation value is larger than the threshold value. Then, the process proceeds to step S24. In step S24, the consistency calculation unit 29 registers the detected noun phrase and the meaning symbol candidate in the noun phrase list. FIG. 13A shows an example of a noun phrase list. In the noun phrase list, the noun phrase candidate character string specified in step S14 is stored in the “extracted noun phrase” column, and the semantic symbol candidate specified in step S22 is stored in the “semantic symbol” column. Stored. Thereafter, the process proceeds to step S26. In addition, the case where step S22 is affirmed means that the part extracted as a noun phrase candidate character string was a correct noun phrase. On the other hand, if the determination in step S22 is negative, the process proceeds to step S26 without passing through step S24. In addition, the case where step S22 is denied means that the part extracted on the assumption that it is a noun phrase candidate character string was not a correct noun phrase.

  If transfering it to step S26, the consistency calculation part 29 will judge whether all the noun phrase candidate character strings were specified. If the determination in step S26 is negative, the process returns to step S14, and the processing / determination of steps S14 to S26 is executed for the next noun phrase candidate character string. For example, as shown in FIG. 14A, it is assumed that a noun phrase candidate character string having a noun phrase candidate ID = CANDI-3 is specified (S14). In this case, as shown in FIG. 14 (b), among the appearing original sentence IDs of the noun phrase candidate ID = CANDI-3, the sentence of the ID not included in the appearing original sentence ID of the noun phrase candidate ID = CANDI-2 (original sentence) [Bacteria] is specified as a semantic symbol of the noun phrase candidate character string by converting the noun phrase candidate character string of ID = Q307, Q308) into a provisional noun [**]. Thereby, the second data shown in FIG. 13B is added to the noun phrase list.

  Thereafter, when the determination in step S26 is affirmed, the process proceeds to step S28, where the syntax analysis unit 30 replaces the noun phrase with a semantic symbol based on the noun phrase list, and analyzes the document using the parsed corpus 40. To do. As a result, the syntax analysis unit 30 can analyze a sentence in which noun phrases are appropriately separated and replaced with appropriate semantic symbols, so that a highly accurate syntax analysis result can be obtained. Thereafter, the syntax analysis unit 30 proceeds to step S30 and outputs the analysis result to the translation unit 52 via the output unit 34.

  Note that the translation unit 52 can translate a document using a highly accurate syntax analysis result. This makes it possible to obtain a highly accurate translation result.

  As can be seen from the above description, in the processing of steps S14 to S22 in FIG. 4, among the syntax examples included in the parsed corpus 40, syntax examples similar to a plurality of sentences including noun phrase candidate character strings are included. Each is extracted (see S66, FIG. 11A), and in steps S18, S20, and S22, based on the extraction results, it is determined whether the extraction of the noun phrase candidate character string is correct, and the meaning of the noun phrase candidate character string is specified. It can be said that. That is, by the consistency calculation unit 29 according to the present embodiment, an extraction unit that extracts, from among a plurality of correct syntaxes included in the parsed corpus 40, syntaxes similar to a plurality of sentences including noun phrase candidate character strings, and A function as a determination unit that determines whether or not the noun phrase candidate character string is a correct noun phrase based on the extraction result by the extraction unit is realized. In addition, the consistency calculation unit 29 of the present embodiment realizes a function as a specifying unit that specifies the meaning of the noun phrase candidate character string based on the syntax extracted by the extraction unit.

  As described above in detail, according to the first embodiment, in the analysis unit 50, the noun phrase candidate extraction unit 22 extracts a word group that appears multiple times in the document as a noun phrase candidate character string ( S12), the consistency calculation unit 29 extracts syntax examples similar to a plurality of sentences including noun phrase candidate character strings from among a plurality of syntax examples included in the parsed corpus 40, and each of the plurality of sentences Based on the extracted syntax example, it is determined whether or not the extraction of the noun phrase candidate character string is correct (S14 to S22). As a result, by assuming a noun phrase candidate character string using a plurality of sentences and determining whether the assumption is correct, it is possible to accurately determine whether the noun phrase candidate character string is a correct noun phrase in the entire sentence. Can judge well. In this case, for example, a noun phrase candidate character string is assumed in one sentence, and it is determined whether or not the noun phrase candidate character string is a correct noun phrase with higher accuracy than the method of comparing with the syntax example of the parsed corpus 40. be able to. Thereby, since there is no word form conversion by part of speech, it is possible to obtain a highly accurate translation result in Chinese translation that is likely to be mistranslated due to ambiguity of part of speech.

  In the first embodiment, a noun phrase candidate having a large number of characters in the noun phrase candidate character string is preferentially processed (S14). As a result, it is possible to avoid occurrence of a situation where a noun phrase having a large number of characters is divided, which occurs when a noun phrase candidate character string having a small number of characters is processed first, and to realize a highly accurate syntax analysis.

  In the first embodiment, the consistency calculation unit 29 identifies the meaning of the noun phrase candidate character string based on the syntax extracted for each of the plurality of sentences (determines the meaning symbol, and determines the noun phrase list). To register). As a result, it is possible to perform syntax analysis in consideration of the meaning of the noun phrase, thereby enabling high-accuracy syntax analysis.

  In the first embodiment, since the translation unit 52 translates the document based on the analysis result of the analysis unit 50, a high-precision translation result is obtained by translation based on the high-precision syntax analysis. Can do.

<< Second Embodiment >>
Hereinafter, the second embodiment will be described in detail with reference to FIGS. In the second embodiment, an example in which a plurality of noun phrase candidate character strings are included in one sentence will be described. The device configuration of the translation terminal 10 is the same as that of the first embodiment, except that the analysis unit 50 has a function as the noun phrase set temporary storage unit 25 as shown in FIG. .

  FIG. 17 is a flowchart showing an outline of processing of the analysis unit 50 in the second embodiment. In FIG. 17, processes different from those of the first embodiment are indicated by adding “′” to the step number.

  In the process of FIG. 17, when a translation target (analysis target) document is input, the noun phrase candidate extraction unit 22 executes a noun phrase candidate extraction process in step S12, as in the first embodiment. (See FIG. 6). In the second embodiment, it is assumed that a patent document as shown in FIG. 18 is input. In step S12, the underlined portion in FIG. 19A is extracted as a noun phrase candidate character string. It shall be assumed. In this case, it is assumed that the data of FIG. 19B and FIG. 19C are temporarily stored in the noun phrase candidate temporary storage unit 26.

  Next, in step S13 'of FIG. 17, the noun phrase candidate extraction unit 22 executes a subroutine of noun phrase set candidate extraction processing. Specifically, in step S13 ', the process shown in FIG. 20 is executed.

  In the process of FIG. 20, in step S <b> 102, the noun phrase candidate extraction unit 22 sets the parameter i of the noun phrase candidate to 1 and sets the parameter k of the sentence to 1. Next, in step S104, the noun phrase candidate extraction unit 22 identifies the noun phrase candidate Ni (= N1). Here, it is assumed that a noun phrase candidate character string of noun phrase candidate ID = CANDI-1 is specified. And the noun phrase candidate extraction part 22 confirms whether the specified noun phrase candidate exists in the sentence zk (= z1). Note that the sentence z1 is, for example, the original sentence ID = Q302 in FIG.

  Next, in step S106, the noun phrase candidate extraction unit 22 determines whether i is the maximum value of i (imax = 2). If the determination here is negative, in step S108, the noun phrase candidate extraction unit 22 increments i by 1 (i ← i + 1), and returns to step S104.

  Returning to step S104, the noun phrase candidate extraction unit 22 specifies the noun phrase candidate character string of the noun phrase candidate ID = CANDI-2 as the noun phrase candidate N2, and whether the noun phrase candidate character string exists in the sentence z1. Confirm whether or not. Thereafter, when the determination in step S106 is affirmed, the process proceeds to step S110, and the noun phrase candidate extraction unit 22 determines whether one or more noun phrase candidate character strings exist in the sentence zk (= z1). .

  When the determination in step S110 is affirmed, the process proceeds to step S112, and the noun phrase candidate extraction unit 22 uses the noun phrase candidate character string information that was present in the sentence zk in the noun phrase set temporary storage unit 25 as a noun phrase. Store as set information. FIG. 19D shows an example of data temporarily stored by the noun phrase set temporary storage unit 25. As shown in FIG. 19D, the data temporarily stored in the noun phrase set temporary storage unit 25 includes items of “noun phrase set ID” and “noun phrase set”. In the case of the sentence zk (ID = Q302), since both noun phrase candidate IDs = CANDI-1 and CANDI-2 exist, in step S112, as a noun phrase set, as shown in noun phrase set ID = 1 Information will be stored. Note that the noun phrase candidate extraction unit 22 does not store the information already stored in the noun phrase set temporary storage unit 25 in duplicate.

  After step S112 or when the determination in step S110 is negative, the process proceeds to step S114. In step S114, the noun phrase candidate extraction unit 22 determines whether k is the same as the maximum value (kmax) of k. If the determination is negative, the noun phrase candidate extraction unit 22 returns i to 1 and increments k by 1 in step S116, and then returns to step S102. Thereafter, the processes after step S104 are executed until the determination at step S114 is affirmed. Then, the data as shown in FIG. 19D is stored in the noun phrase set temporary storage unit 25, and the processing of FIG. 20 is terminated when the determination in step S114 is affirmed. Thereafter, the process proceeds to step S14 'in FIG.

  In step S14 'of FIG. 17, the noun phrase replacement unit 24 identifies one noun phrase set candidate from FIG. 19D. For example, it is assumed that the noun phrase replacement unit 24 identifies a noun phrase set candidate with the noun phrase set ID = 1 from FIG.

  Next, in step S16 ', the noun phrase replacement unit 24 replaces the noun phrase candidate character string in the document with a provisional noun. In this case, the noun phrase candidate character strings included in the noun phrase set candidates identified in step S14 ′ are replaced using provisional nouns [** 1] and [** 2] as shown in FIG. To do.

  Next, in step S18 ', the consistency calculation unit 29 executes a semantic symbol candidate extraction process for the noun phrase candidate character string. In this step S18 ′, each sentence in FIG. 21A is parsed as shown in FIG. 21B, and a search is made as to whether or not a syntax structure example similar to each syntax structure is included in the parsed corpus 40. To do. For example, it is assumed that the sentence (Q302) shown in FIG. 22A is similar to the syntax structure case (C1001, C1002) shown in FIG. In this case, as shown in FIG. 22C, semantic symbol candidates [core] and [bacteria] are stored in the semantic candidate list of [** 1], and semantic symbols are stored in the semantic candidate list of [** 2]. Candidate [soil] and “metal” are saved. Hereinafter, similarly, parsing is also performed on other sentences, but it is assumed that the semantic candidate list remains as shown in FIG.

  Next, in step S19, the consistency calculation unit 29 determines whether or not a semantic symbol candidate has been extracted. If the determination is negative, the process proceeds to step S26 '. If the determination is positive, the process proceeds to step S20'.

  In step S20 ', the consistency calculation unit 29 executes a process for evaluating consistency when the noun phrase set candidate is a semantic symbol candidate. Here, as shown in FIG. 22 (d), the consistency calculation unit 29 makes hypotheses (hypotheses 1 to 4) of the provisional nouns based on the semantic candidate list in FIG. Similar to the embodiment, the consistency evaluation value is calculated. For example, as a result of syntactic analysis when hypothesis 1 is adopted, it is assumed that a matching case exists in the parsed corpus 40 for only the sentence with the original sentence ID = Q308 as shown in FIG. In this case, the consistency evaluation value of Hypothesis 1 is 1/4 = 0.25. Similarly, as a result of the syntax analysis when the hypothesis 2 is adopted, it is assumed that a matching case exists in the parsed corpus 40 only for the sentence with the original sentence ID = Q302 as shown in FIG. In this case, the consistency evaluation value of Hypothesis 2 is 1/4 = 0.25. Further, as a result of the syntax analysis when the hypothesis 3 is adopted, as shown in FIG. 23C, it is assumed that a matching case exists in the sentences of the original sentence ID = Q302, Q304, and Q308. In this case, the consistency evaluation value of Hypothesis 3 is 3/4 = 0.75. Further, as a result of the syntax analysis when hypothesis 4 is adopted, it is assumed that a matching case exists only in the sentence with the original sentence ID = Q304 as shown in FIG. In this case, the consistency evaluation value of Hypothesis 4 is 1/4 = 0.25.

  As described above, when the consistency evaluation values shown in FIGS. 23A to 23D are obtained in step S20 ', the consistency calculator 29 proceeds to the next step S22'.

  In step S22 ', it is determined whether or not there is a hypothesis whose consistency evaluation value is the maximum and exceeds a threshold value (for example, 0.5). If the determination is negative, the process proceeds to step S26 '. If the determination is positive, the process proceeds to step S24. When the process proceeds to step S24, the consistency calculating unit 29 registers the semantic symbols in the noun phrase list as illustrated in FIG. 24, and then proceeds to step S26 '.

  In step S26 ', the consistency calculation unit 29 determines whether all noun phrase set candidates have been specified. If the determination in step S26 'is negative, the process returns to step S14' and the above-described processing is repeated. In addition, like the noun phrase set ID = 2, 3 noun phrase set in FIG. 19D, the noun phrase candidate characters whose semantic symbols have already been determined in the previous process (process of noun phrase set ID = 1). For a set including only columns, the processing after step S14 ′ need not be executed. On the other hand, if the determination in step S26 'is affirmative, the process proceeds to step S28.

  In step S28, as in the first embodiment, the syntax analysis unit 30 performs document analysis by replacing noun phrases with semantic symbols based on the noun phrase list (FIG. 24). In step S <b> 30, the syntax analysis unit 30 outputs the analysis result of step S <b> 28 to the translation unit 52 via the output unit 34.

  As described above in detail, according to the second embodiment, the same effect as that of the first embodiment can be obtained, and when there is a sentence including a plurality of types of noun phrase candidate character strings, The phrase candidate extraction unit 22 identifies a sentence including at least one of a plurality of types of noun phrase candidate character strings from the document, and the consistency calculation unit 29 parses a syntax example similar to each of the identified sentences. Extract from the finished corpus 40. Then, the consistency calculation unit 29 determines whether or not the plurality of noun phrase candidate character strings are correct noun phrases based on the extracted syntax examples. As a result, it can be determined at a time whether or not a plurality of types of noun phrase candidate character strings are correct noun phrases, so that a comprehensive determination using a plurality of types of noun phrase candidate character strings can be performed. . As a result, it is possible to determine with higher accuracy than when determining whether the noun phrase candidate character strings are correct noun phrases one by one.

  In the first and second embodiments, the case where the translation terminal 10 includes the analysis unit 50 and the translation unit 52 has been described. However, the present invention is not limited to this. For example, as illustrated in FIG. 25, the server 110 connected to the network 180 may include an analysis unit 50 and a translation unit 52. In this case, by inputting the document to be translated from the client 120, the document is translated with high accuracy in the server 110, and the translated sentence is output from the server 110 to the client 120. In the case of FIG. 25, the client 120 may have one of the analysis unit 50 and the translation unit 52.

  In the first and second embodiments, the translation from Chinese to Japanese has been described as an example. However, the present invention is not limited to this. In the translation of languages other than Chinese that have no change in word form due to part of speech, the devices and methods of the first and second embodiments may be used. Further, when translating from Chinese or the like into a language other than Japanese, the devices and methods of the first and second embodiments may be used.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium (except for a carrier wave).

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, regarding the above description of the first and second embodiments, the following additional notes are disclosed.
(Supplementary note 1) An assumption part that assumes a group of words appearing multiple times in a document as a noun phrase candidate;
An extraction unit that extracts a plurality of correct syntaxes included in the syntax list, each of which is similar to a plurality of sentences including the noun phrase candidate,
A document analysis device comprising: a determination unit that determines whether or not the assumption of the noun phrase candidate is correct based on an extraction result by the extraction unit.
(Supplementary Note 2) When there are a plurality of noun phrase candidates,
The document analysis apparatus according to appendix 1, wherein the extraction unit preferentially selects noun phrase candidates having a large number of characters, and extracts syntaxes similar to a plurality of sentences including the selected noun phrase candidates. .
(Additional remark 3) The document analysis apparatus of Additional remark 1 or 2 further provided with the specific | specification part which specifies the meaning of the said noun phrase candidate based on the syntax which the said extraction part extracted.
(Supplementary Note 4) When a sentence including plural types of noun phrase candidates exists in the document,
The extraction unit identifies a plurality of sentences including at least one of the plurality of types of noun phrase candidates from the document, extracts a syntax similar to each of the plurality of identified sentences from the syntax list,
The document analysis according to any one of appendices 1 to 3, wherein the determination unit determines whether or not the assumptions of the plurality of types of noun phrase candidates are correct based on an extraction result by the extraction unit. apparatus.
(Supplementary note 5) The document analysis device according to any one of supplementary notes 1 to 4, further comprising a translation unit that performs translation of the document based on a determination result of the determination unit.
(Supplementary Note 6) Assuming a word group that appears multiple times in a document as a noun phrase candidate,
Of the plurality of correct syntaxes included in the syntax list, respectively extract syntaxes similar to the plurality of sentences including the noun phrase candidates,
Based on the extracted results, determine whether the noun phrase candidate assumption is correct,
A document analysis program that causes a computer to execute processing.
(Supplementary Note 7) When there are a plurality of noun phrase candidates,
The document analysis according to appendix 6, wherein the extracting process preferentially selects a noun phrase candidate having a large number of characters and extracts a plurality of sentences similar to a plurality of sentences including the selected noun phrase candidate. program.
(Additional remark 8) The document analysis program of Additional remark 6 or 7 which makes the said computer further perform the process which specifies the meaning of the said noun phrase candidate based on the syntax extracted in the said process to extract.
(Supplementary Note 9) When there is a sentence including plural types of noun phrase candidates in the document,
In the extracting process, a plurality of sentences including at least one of the plurality of types of noun phrase candidates are identified from the document, and syntax similar to each of the identified plurality of sentences is extracted from the syntax list. ,
The determination process determines whether or not the assumptions of the plurality of types of noun phrase candidates are correct based on an extraction result in the extraction process. Document analysis program.
(Supplementary note 10) The document analysis program according to any one of supplementary notes 6 to 9, which causes the computer to execute a process of translating the document based on a determination result in the determination process.
(Supplementary Note 11) Assuming a word group that appears multiple times in a document as a noun phrase candidate,
Of the plurality of correct syntaxes included in the syntax list, respectively extract syntaxes similar to the plurality of sentences including the noun phrase candidates,
Based on the extracted results, determine whether the noun phrase candidate assumption is correct,
A document analysis method in which processing is executed by a computer.
(Supplementary Note 12) When there are a plurality of noun phrase candidates,
12. The document analysis according to appendix 11, wherein the extracting process preferentially selects noun phrase candidates having a large number of characters, and extracts syntaxes similar to a plurality of sentences including the selected noun phrase candidates. Method.
(Supplementary note 13) The document analysis method according to supplementary note 11 or 12, wherein the computer further executes a process of specifying the meaning of the noun phrase candidate based on the syntax extracted in the extracting process.
(Supplementary Note 14) When there is a sentence including plural types of noun phrase candidates in the document,
In the extracting process, a plurality of sentences including at least one of the plurality of types of noun phrase candidates are identified from the document, and syntax similar to each of the identified plurality of sentences is extracted from the syntax list. ,
The determination process determines whether or not the assumptions of the plural types of noun phrase candidates are correct based on an extraction result in the extraction process. Document analysis method.
(Supplementary note 15) The document analysis method according to any one of supplementary notes 11 to 14, wherein the computer further executes a process of translating the document based on a determination result in the determination process.

10 Translation terminal (document analysis device)
22 Noun phrase candidate extraction part (assuming part)
29 Consistency calculation unit (extraction unit, determination unit, identification unit)
40 parsed corpus (syntax list)
52 Translation Department

Claims (7)

A hypothesis that assumes words that appear multiple times in the document as noun phrase candidates;
An extraction unit that extracts a plurality of correct syntaxes included in the syntax list, each of which is similar to a plurality of sentences including the noun phrase candidate,
A document analysis apparatus comprising: a determination unit that determines whether or not the assumption of the noun phrase candidate is correct based on whether the extraction unit has extracted syntax similar to a plurality of sentences including the noun phrase candidate; . When there are a plurality of noun phrase candidates,
2. The document analysis according to claim 1, wherein the extraction unit preferentially selects noun phrase candidates having a large number of characters, and respectively extracts syntaxes similar to a plurality of sentences including the selected noun phrase candidates. apparatus.   The document analysis apparatus according to claim 1, further comprising: a specifying unit that specifies the meaning of the noun phrase candidate based on the syntax extracted by the extraction unit. When there is a sentence including a plurality of types of noun phrase candidates in the document,
The extraction unit identifies a plurality of sentences including at least one of the plurality of types of noun phrase candidates from the document, extracts a syntax similar to each of the plurality of identified sentences from the syntax list,
The determination unit is correct in assuming the plurality of types of noun phrase candidates based on whether the extraction unit has extracted a syntax similar to a plurality of sentences including at least one of the plurality of types of noun phrase candidates. The document analysis apparatus according to claim 1, wherein the document analysis apparatus determines whether or not.   The document analysis apparatus according to claim 1, further comprising a translation unit that performs translation of the document based on a determination result of the determination unit. Assuming a group of words that appear multiple times in the document as noun phrase candidates,
Of the plurality of correct syntaxes included in the syntax list, respectively extract syntaxes similar to the plurality of sentences including the noun phrase candidates,
Determining whether or not the noun phrase candidate assumption is correct based on whether or not a syntax similar to a plurality of sentences including the noun phrase candidate can be extracted in the extracting process;
A document analysis program that causes a computer to execute processing. Assuming part assumes a group of words appearing multiple times in the document as a noun phrase candidate,
The extraction unit extracts a plurality of correct syntaxes included in the syntax list, respectively, and extracts a syntax similar to a plurality of sentences including the noun phrase candidate,
The determination unit determines whether the assumption of the noun phrase candidate is correct based on whether the extraction unit has extracted a syntax similar to a plurality of sentences including the noun phrase candidate.
A document parsing method that performs processing.

Images