JPH02254565A - Syntax analysis system - Google Patents

Abstract

PURPOSE:To improve the syntax analysis efficiency by performing syntax analysis after limiting parts of speech of words to parts of speech, which satisfy combinations of the highest connection probability, to resolve the polysemy of words having many parts of speech at the time of analyzing these words. CONSTITUTION:A morpheme analyzing part 7a of a translation main body part (translation part) 7 refers to a dictionary for an input text, and a syntax analyzing part 7b gets information of individual words and performs purging in accordance with a grammatical rule, and a tree structure is generated from analysis results. A converting part 7c transforms the tree structure of the input language to that of the output language, and a generating part 7d translates every node of the obtained tree structure. Parts of speech of respective words are limited to parts of speech satisfying combinations of the highest connection probability by calculating the product of connection probability between parts of speech, and syntax analysis is performed after the polysemy of words having many parts of speech is resolved at the time of analyzing these words. Thus, the syntax analysis efficiency is improved.

Description

[Detailed description of the invention] 1 supervisor mutually The present invention relates to a syntax analysis method in natural language processing.

As a prior art related to the present invention, Japanese Patent Application Laid-Open No. 61-4067
2 and Japanese Patent Application Laid-open No. 1-74069. The invention described in Japanese Patent Application Laid-Open No. 61-40672 relates to a multi-part side resolution processing method, in which a multi-part side cancellation rule for determining parts of speech is prepared, and one
The two parts of speech are determined. Also, JP-A No. 61-4067
The invention described in Publication No. 2 relates to an interactive translation method, in which when a human specifies the part of speech of a word in an input sentence,
The translation is based on that.

Processing multiple parts of speech is a very difficult problem in natural language parsing. When one word has multiple meanings, various analysis results can be derived, but anything other than the correct answer is an incorrect analysis.

The disambiguation of multipart speech words has two meanings for syntactic analysis. One is to improve analysis accuracy. Misanalysis is greatly reduced by disambiguation. The other is improving analysis accuracy.

Since other parts of speech are excluded, little analysis is required other than the one that leads to the correct answer. For example, suppose that eight possible solutions are obtained when a sentence with multiple parts of speech is analyzed as is. If multiple parts of speech are eliminated, the number of solution candidates will be greatly reduced, so the number of applications of parsing rules and the number of combinations will be reduced, and the burden of parsing will be significantly reduced.

Mame-15 This invention was made in view of the above-mentioned circumstances.
By introducing a new concatenation probability and using word priority, we aim to improve syntactic analysis efficiency, achieve more accurate syntactic analysis, and provide a syntactic analysis method that can also handle idiom processing. It was made for the purpose of

(1) In a natural language analysis system such as machine translation, the present invention calculates the parts of speech of each word after the morphological analysis section looks up the text to be analyzed in a dictionary. , by calculating the product of the conjunctive probabilities between each, the ambiguity of multi-part speech words is resolved when words with multiple parts of speech are analyzed by limiting the combination to those that satisfy the highest conjunctive probability. (2) In a natural language analysis system such as machine translation, after the morphological analysis unit looks up the text to be analyzed in a dictionary, the part of speech of each word is determined by comparing the part of speech between each word. By calculating the product of conjunctive probabilities, when limiting the combination to those that satisfy the highest concatenating probability, the priority of each part of speech is also calculated as a coefficient of the product, and polysemy of multi-part speech words can be reduced. (3) In a natural language analysis system such as machine translation, after the morphological analysis unit looks up the text to be analyzed in a dictionary, the part of speech of each word is determined by comparing the part of speech between each word. By calculating the product of conjunctive probabilities, when limiting to combinations that satisfy the highest concatenating probability, if there is an idiom among the words looked up in the dictionary, parallel calculation and exclusion are performed depending on the priority of the idiom. It is characterized by performing syntactic analysis by performing calculations and eliminating ambiguity in multipart speech words. Hereinafter, the present invention will be explained based on examples.

FIG. 1 is a block diagram for explaining one embodiment of a translation device using a syntactic analysis method according to the present invention, and in the figure, 1 is a CR
T, 2 is the keyboard, 3 is 0CR14 is the input document, 5 is the spell check section, 6 is the pre-editing section, 7 is the translation main section, 8
9 is a post-editing section, 9 is a dictionary, 10 is a grammar rule, 11 is an output document, and 12 is a printer. Input sentences obtained by file input, keyboard input, or OCR input can be preprocessed using spell check section 5 and preediting section 6.

The output sentence obtained by the translation main unit 7 can be edited by the post-editing unit 8 using translation information. The input and output sentences can be printed using printer 12.

FIG. 2 shows the processing flow of the translation main unit 7. This translation main unit (translation unit) 7 is roughly divided into four processes: morphological analysis, syntactic analysis, conversion, and generation. Then, the input text is looked up in a dictionary, and the syntactic analysis unit 7b obtains information on each word and performs parsing according to grammatical rules, and creates a tree structure from the analysis results. Conversion section 7
In step c, the tree structure of the input language is transformed into the tree structure of the output language, and in the generation section 7d, the obtained tree structure is translated node by node.

The present invention belongs to the above-mentioned syntax analysis section, and here, the input text is assumed to be English text. The morphological analysis unit 7a performs a dictionary lookup on the input text, obtains the result of the dictionary lookup, and proceeds to the syntactic analysis unit 7b. Syntax analysis section 7
In step b, multi-part-of-speech resolution processing is first performed. The multi-part-of-speech resolution method here uses the method introduced in the following literature by S., J., and Derose.

Computational Linguistics
, Vol. 14. No, 1゜su1nter 198
8. p31-39 “Grammatical Cat
egory Disambiguation bySt
physical optimization”S,
J, Derose (BrownUniv,) VOLSU proposed by the author of the above document (Deross)
The analysis section is introduced using a multi-part-of-speech resolution method called NGA. VOLSUNGA has the following features.

■Based on a complete mathematical algorithm, temporary additions are kept to a minimum.

The definition of an optimal part-of-speech string is one in which the product of the part-of-speech conjunction probability and the relative part-of-speech probability that make up the part-of-speech string is maximum.

■Efficient optimal part-of-speech sequence search method (dynamic programming method)
This overcomes exponential calculations.

The parts of speech here refer to detailed classifications of parts of speech, with a total of 100 parts of speech.
There are several types.

Necessary data include a dictionary with part-of-speech classification, relative part-of-speech probability, and idiom priority information (Figure 4), and a part-of-speech conjunctive probability table (Figure 2).
There is a table).

The method uses a dynamic programming method to find the optimal combination of parts of speech from Table 1 below. FIG. 3 shows a flowchart for selecting an optimal part-of-speech sequence.

Table 1: If the optimal part-of-speech combination from To,1 to Tn,j passes through Tn-1,i, then the part up to Tn-1,i is the optimal part-of-speech combination from TO21 to Tn-1,i. It is.

This is because if the part up to Tn-1,i is not optimal, if the optimal part-of-speech combination is selected for the part up to Tn-1,i,
That would be optimal.

Therefore, the optimal part-of-speech combination from TO,1 to Tn,j is the optimal part-of-speech combination from TO21 to Tn-1,i for each 1 (=1..., io) and the optimal part-of-speech combination from Tn-1,i to Tn, It is the optimal one among the combinations to j. For example, The man 5till 5ati her.

Calculate the optimal part-of-speech combination for the sentence.

Assume that each word has the following parts of speech.

Furthermore, to simplify the explanation, the relative part-of-speech probabilities of the parts of speech of each word are omitted. Actually, in addition to the conjunction probability, the part-of-speech relative probability is also a coefficient.

The man 5till saw her
AT NN NN NN PP0VB V
B VBD PPS B Here, AT (=article), NN (=noun), ppo (
= pronoun object), pps (= possessive pronoun), RB (=
adverb), VB (=verb), and VBD (=verb past).

There are 11 part-of-speech combinations from "The" at the beginning to "her" at the end.
There are 24 ways, 213m2*2=24. Calculate the optimal combination using the probabilities in Table 2 on the next page.

＾　　　Loco 2　　　〉 QXQ z　　　cQ　　C15 Z　　　　＞　　　＝ === Wife ↑ ≧　　　　α ≧=　　　〉 2 Tomoe ↑ ■ ↑ Lu Table 2 (Example of connection probability) Next, an example of correction using relative part-of-speech probabilities will be shown.

so: QL (limiting side, 932), CS (subordinating conjunction, 479).

LIH (interjection, 1) The number is the part of speech relative probability so tha
The part of speech of the sequence t is estimated using only the conjunction probability and the relative part of speech probability.

Method P(Ull-C3)) P(C
3-C5) Conjunction probability and relative part-of-speech probability P(UH-C3)*P(so-UH)*P(that-
C3)<P (C8-C5)*P (so-C3)*P
(that-C3)P, () indicates probability. P(UH
-C3) indicates the connection probability of tlH-C5. P (so
-C3) indicates the relative probability of the CS of so. The correct answer C3-CS is obtained when the conjunction probability and the relative part-of-speech probability are used together.

Next, an example of processing when there is an idiom is shown.

The 1lan came in order to
Assume that win has the following parts of speech.

The man came in order to
Win.

AT NN VBD PRNN TONN
To (to with infinitive) VB RB VB
PRVB PR (preposition) RB PPN (personal pronoun nominative) <--TO--) The part-of-speech combinations from the first work to the last win are 1*2*
1 * 2 * 2 * 3 * 2 = 48 1 ne 2 * 1 * 1 1 2 = 4 There are 48 ways in total. Normally, an idiom should be considered as a single word rather than treated individually, but for the sake of convenience, each of its constituent words is treated as a provisional word, leaving open the possibility of combinations. At this time, of course, this part cannot be combined with other words. The conjunction probability and part-of-speech relative probability are calculated for the last word constituting the compound word. In this method, the longer the idiom, the fewer the number of coefficients, so it is necessary to multiply the number of coefficients accordingly.

In this example, in order to is an idiom. This part is "1nl)" There are 12 combinations of "order" to, 2*2*3, but "in order t"
There is only one type of "o". Below, it is expressed as this -*-TO. The -ゝ° part indicates that the combination is fixed.

The man came in order
To win.

NN NN NN
PRAT −+ AT 4 AT
VBD −+ AT VBDVB
VB VB RB book NN PRNN NN P
RNN T.

4AT VBD 4AT V
BD PRVB RB VB
VB RB VB RB*
−−＊ ＊ −−＊ −−T
.

NN PRNN To NN4AT
VBD PRVB RB
VB RB VB* --* --T.

The man came in order
To win.

AT NN VBD *--*--To V
BAlso, priority information can be given to phrases in a dictionary. So far, we have explained the case where an idiom does not have priority information. When having priority information, possibilities other than idiomatic words are discarded. In other words, when "in order to" has priority information, "1n""order"to" will not be looked up in the dictionary.The example sentence will only have the following parts of speech.This priority information may be given by an idiom, or it may not be given. There are some things.

The man came in order to
Win.

AT NN VBD <--To ---> N
NVB VB In this way, syntax analysis is performed after eliminating multiple items. Since ambiguity is largely eliminated in syntactic analysis, the main task is to create this structure.

As is clear from the above description, according to the present invention, the parsing rate is improved by newly introducing the concatenation probability according to claim 1. Furthermore, according to claim 2, by using word priority in combination, more accurate syntactic analysis can be performed.

Also according to claim 3.

Supports idiom processing.

[Brief explanation of drawings]

FIG. 1 is a block diagram for explaining an embodiment of a translation device using the syntax analysis method according to the present invention, FIG. 2 is a diagram showing the processing flow of the translation main body in FIG. 1, and FIG. teeth,
FIG. 4, which is a diagram showing the flow of selection by optimal part of speech, is a diagram showing an example of a dictionary. 1...CRT, 2...Keyboard, 3...0CR
14... Input document, 5... Space check section, 6
... Pre-editing section, 7... Translation body section, 8... Post-editing section, 9... Dictionary, 10... Grammar rules, 11... Output document, 12... Printer. Figure Figure

Claims (1)

[Claims] 1. In a natural language analysis system such as machine translation, after the morphological analysis unit looks up the text to be analyzed in a dictionary, the product of the conjunctive probabilities between the parts of speech of each word is calculated. By doing this, syntactic analysis is performed after eliminating the ambiguity of multi-part speech words when a word with multiple parts of speech is analyzed, limiting it to the combination that satisfies the highest conjunctive probability. parsing method. 2. In a natural language analysis system such as machine translation, after the morphological analysis unit looks up the text to be analyzed in a dictionary, the part of speech that each word has is calculated to find the most connected part of speech by calculating the product of the connection probabilities between each part of speech. When restricting to combinations that satisfy a high probability, the priority of each part of speech is also calculated as a product coefficient, and syntactic analysis is performed by eliminating polysemy of multi-part speech words. Parsing method. 3. In a natural language analysis system such as machine translation, after the morphological analysis unit looks up the text to be analyzed in a dictionary, the part of speech that each word has is calculated to find the most connected part of speech by calculating the product of the connection probabilities between each part of speech. When limiting to words that satisfy combinations that increase the probability, if there is a compound word among the words looked up in the dictionary, parallel calculation and exclusive calculation are performed depending on the priority of the compound word to eliminate polysemy of multi-part speech words. A syntactic analysis method characterized by performing syntactic analysis.