JP2632806B2 - Language analyzer - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a language analyzer, and particularly to a language analyzer useful for an automatic translator.

2. Description of the Related Art When a corresponding Japanese sentence is created from a foreign language sentence such as English, for example, the morpheme of the input English sentence is analyzed, its syntax is analyzed, and its sentence structure is converted. Generate a translation of

In the cfg grammar (context-free grammar: context-free grammar), the sentence partially conforms to both the so-called bottom-up analysis method and the top-down analysis method that perform parsing from the end of the sentence to the beginning. An analytical solution is also output. Therefore, a disadvantage is that many useless solutions that cannot be used finally are output. Many of these useless solutions are clearly wrong when humans read them.

If a large number of analysis results including such solutions that are apparently erroneous are obtained by the syntax analysis, the efficiency of the subsequent steps, such as the structure conversion and the translation generation, will be reduced. In other words, structural conversion is performed on a useless solution, or translation generation is performed, and the appropriateness of those results is determined in each processing step, resulting in a waste of processing time.

For example, a preposition phrase or an adverbial phrase in an English sentence may preferably be freely selected in terms of context, that is, a modification destination. So, if those modifiers are not fixed,
A method is conceivable in which all possible parties are considered as solution possibilities. There is also a method of giving priority to dictionary description. In the latter case, if the dictionary is indexed and a description is given that a specific word takes a specific preposition, the co-occurrence relation is always adopted. This risks ignoring the original intent of the statement. For example, the English sentence "I saw a man in
In the example of "the park with a telescope.", if the dictionary word "saw" is described as taking the preposition "with", the preposition phrase "with a telescope" must always take the verb "saw" as the destination. And other possibilities are ignored.

However, if all such possible candidates are solved in the parsing, the number of candidates increases,
The number of solutions generated in the subsequent structure conversion and translation generation processing is enormous. Such a large number of solution candidates results in a significant reduction in the speed of subsequent processing.

Therefore, in order to improve the efficiency of the entire automatic translation process, it is necessary to reduce the number of such useless solutions to increase the efficiency of the analysis and to make the analysis result more reliable.

Objectives In view of such demands, an object of the present invention is to provide a language analysis device capable of efficiently performing syntax analysis.

Configuration In order to achieve the above object, the present invention provides a dictionary unit storing dictionary data for analyzing a sentence in a predetermined language,
Analyzing means for performing morphological analysis by indexing the dictionary means for the input sentence of the predetermined language and performing syntax analysis on the sentence based on the analysis result;
It has a table that defines conditions for selecting the destinations of the components of the language sentence, analyzes the surface structure of the sentence by applying rules to the grammar, and provisionally determines the destinations of the components included in the sentence. Randomly set syntax tree candidates by assigning priorities, selecting high-priority ones from these candidates, setting the priorities of the selected candidates to a condition table, and dictionary data from the dictionary means. Is characterized by a language analysis device that corrects by referring to. Hereinafter, a specific description will be given based on an embodiment of the present invention.

Referring to FIG. 1, there is shown an overall configuration of an embodiment in which a language analyzer according to the present invention is applied to an English-Japanese automatic translator. Needless to say, the present invention is effectively applied not only to an English-Japanese automatic translator that translates English into Japanese, but also to an automatic translator that translates one language into another language.

This embodiment has an input unit 10 for inputting an English text 12 to be translated into Japanese. The input unit 10 may be, for example, a keyboard having character keys such as alphanumeric keys and function keys, an optical character reader (OCR) for reading English text recorded on paper, and / or a storage medium such as a magnetic disk. It may include a file storage device that reads the recorded English text.

The English text input by the input unit 10 is
It is read into 14 and pre-processing for translation is performed. Here, recognition of sentences and processing of unknown words are mainly performed. This functions as part of the morphological analysis.

The pre-edited English sentence data is transferred to the morphological analysis unit 16 together with the information obtained by the pre-editing. The morphological analysis unit 16 indexes the word dictionary 18 and divides the sentence into sentences, analyzes the morphemes of the English sentence, performs various processing such as processing of unknown words, proper nouns, expressions of time, expressions of numbers, etc. , And performs the processing of the entire sentence such as recognition of the same rank. The morphological analysis rules are stored in the analysis rule file 36.

The English sentence data thus morphologically analyzed is transferred to the syntactic analysis unit 20 together with the dictionary information obtained by the morphological analysis. In this embodiment, the syntax analysis I unit 20 applies bottom-up, right-to
This is a functional unit that analyzes the surface structure on the left and finds all syntactic possibilities.

The English sentence data parsed (parsed) by the parsing I unit 20 is sent to the parsing II unit 22 together with the parsing information. Here, a solution is selected by applying the structural description from the surface parsing result by the syntax analysis I. This creates a probable parse tree of the English sentence and creates its structure. These parsing rules are still in parsing rule file 36
Is stored in

The parsed English sentence data is transferred to the structure conversion unit 24 as parse tree data. The structure conversion unit 24 creates a syntax tree of a corresponding Japanese sentence from a syntax tree that is an intermediate structure of the English sentence, and converts the Japanese sentence into a Japanese base structure that is easy to translate.

The syntax tree data indicating the Japanese syntax tree whose structure has been converted in this manner is sent to the translated sentence generating unit 26, and the translated sentence is generated by the latter. This is a function for generating a Japanese sentence from a Japanese base structure. First, in order to match the word order with that of Japanese, a syntax generation that changes the tree structure by rearranging the order is performed, and then a morpheme generation is performed, and the translated text is translated into top-down, left-to-right in the syntax tree. Generate

The translated sentence data, that is, the translated sentence data, is sent to the post-editing unit 30. The post-editing unit 30 uses the information used for the translation process to index the dictionary 18 to correct the translated sentence data, thereby completing a more natural Japanese sentence. This Japanese sentence data is transferred to the output unit 32 and the translated Japanese sentence 34
Is output from the output unit 32. The output unit 32 includes a file storage device such as a printer, a display, and / or a magnetic disk.

The flow of these series of translation processes is controlled by the control unit 38 that controls the entire control of the present apparatus. In the present embodiment, the word dictionary 18 stores dictionary data for English and Japanese words, and describes not only vocabulary but also various information such as relations, that is, co-occurrence relations, meanings, singularity, and parts of speech. ing. The analysis rule file 36 stores rule data for morphological analysis and syntax analysis.

The operation display unit 40 is connected to the control unit 38. The operation display unit 40 gives various instructions from the operator to the apparatus.
For example, it has operation keys such as a translation instruction key and a cursor key, a display and an indicator for visually displaying input English sentence text, translation result Japanese sentence, intermediate data such as dictionary information, various instructions to an operator, and the like. Many of these operation display functions are provided on the keyboard when the input unit 10 has a keyboard, and the output unit.
If the display 32 is provided with a display, the display may be included in the display.

By the way, the parsing I unit 20 converts the morphologically analyzed English text data into a cfg grammar rule in the English text as a bottom-up, ri.
Apply to ght-to-left to derive all possible syntactic solutions for the sentence. This solution is generally understood in the form of a structural tree. This means that for each sentence, the words or phrases contained in it are related to each other by subordination or co-occurrence such as qualification and case, and
It shows mutual dependency such as children and grandchildren. Each word or phrase occupies a node or node in the structure tree.

This embodiment may be configured so as to identify the morphological and lexical features of the sentence and determine the syntactic unit before the syntax analysis. These syntactic units are referred to herein as "units" and "blocks". The function of recognizing the syntactic unit of the input English sentence as a block is performed by the morphological analysis unit 16.

A “unit” is a group of words that is the minimum unit of the translation process. At the time of parsing, this is treated as one word, and the dictionary information of each component included in the word is not used.

A "block" is a syntactic unit that gives priority to the analysis inside the block over the analysis outside the block, and treats the block outside the block as equivalent to a unit. For example, it may correspond to an intermediate symbol used in the cfg grammar, in addition to a clause, a phrase, and the like. It can also be nested. That is, a block may be further included in the block. Further, the concept of a block, including a sentence, a paragraph, and a whole sentence, may be regarded as one block. This process of prioritizing the partial analysis is referred to herein as "partial parse". by this,
The above-mentioned useless parsing is reduced, the efficiency of analysis is improved, and a more reliable analysis result is obtained.

The word dictionary 18 stores dictionary information on English words and idioms. In this embodiment, an entry is formed for each variation of each word, and all the information is expanded. For example, the part-of-speech information can have information on a plurality of parts of speech.

The analysis rule file 36 stores data of a head condition indicating the head of the block and an end condition indicating the end of the block as a table, and also stores data of a cfg grammar rule and structure description. These are used, for example, in a provisional tree structuring process 204 (FIG. 3) described later.

Meanwhile, the morphological analysis unit 16 first divides the English sentence input from the pre-editing unit 14 into sentences that are translation units. At that time, spelling errors and unregistered words are detected. The dictionary 18 is indexed for each sentence, and dictionary information of each component is fetched. Various grouping processes are performed according to the dictionary information.

In the morphological analysis unit 16, in addition to the block recognition, for example, proper nouns, derivatives, unknown words, abbreviations, numbers, time expressions,
It performs various processing such as processing of hyphen, apostrophe "'", estimation of equality, processing of additional questions, etc.
Create morphological analysis data. The morphologically analyzed English sentence data is transferred to the syntax analysis unit 20 together with the dictionary information obtained by the morphological analysis.

The syntactic analysis unit 20 applies the cfg grammar rule to the English sentence data, analyzes the surface structure of the sentence bottom-up, right-to-left, and finds all syntactic possibilities. That is, by applying the context-free grammar rules stored in the analysis rule file 36, the surface structure of the English sentence is analyzed, and all possible syntax trees are found (111, FIG. 2). At this time, if a block is included, the above-described partial parse is performed, and local analysis is prioritized. This improves the efficiency and accuracy of the analysis.

When the surface structure data defining the syntactic unit and the subordinate relation is obtained, it is sent to the parsing II unit 24. This data is a list of cfg analysis results (WFS) shown in Fig. 3.
It takes the form of 200 and is easily understood in the form of a parse tree as described above. The parsing II unit 22 applies a structural description from the surface parsing result of the parsing I unit 20 in consideration of the weight or priority and selects a solution. In this way, a probable parse tree candidate for the English sentence is created and its structure is created (112). Furthermore, the destination of the prepositional phrase or the adverbial phrase is determined using a dictionary description or the like, and a top priority table is created. As a result, a more likely parse tree is obtained (113
~ 115).

Referring to FIG. 3, the English text “I saw a man in the park wi
th a telescope. "will be described as an example. The syntactic element of this English sentence is NP1- (Vt1 or Vt2) -NP2-PP1-PP2, where NP is a noun phrase, Vt is a transitive verb, PP is a prepositional phrase, and subscripts indicate different vocabulary or forms, for example, Vt1 symbolizes the past tense of the verb "see" and Vt2 symbolizes the present tense of the verb "saw". .

The parsing I unit 20 analyzes the surface structure of the English sentence as SE1 ← NP1 VP1 PP1 PP2 SE2 ← NP1 VP2 PP1 PP2 VP1 ← Vt1 NP2 VP2 ← Vt2 NP2. Of course, other than these can also be obtained.
However, SE indicates a sentence, and VP indicates a verb phrase. This analysis result data is stored in the cfg analysis result list 200 by the syntax analysis unit 22.

The parsing II unit 22 first indexes the dictionary 18 according to the list 200 of the cfg analysis result obtained from the parsing I unit 20, and executes a provisional tree structuring process. The cfg grammar rules stored in the analysis rule file 26 include data defining the priority of selection in accordance with the structure of the sentence.

In the present embodiment, for example, a syntax of SE ← NP NP [PP] n is defined in the cfg rule as priority “4”. Here, n is an arbitrary integer of 0 or more. [] Is a symbol indicating morpheme subordination or co-occurrence. This syntactical expression means that the sentence structure is [VP [NP PPa .....]], and that n temporary provisional preposition phrases PPa are included. However, the suffix “a” indicates that the destination is a node whose provisional destination is temporarily determined.

Also, a syntax such as VP ← Vt NP is specified in the cfg rule as priority “3”. This sentence structure is [Vt [NP]]. In addition, syntax such as SE ← NP VP priority “10” VP ← V NP priority “5” is also specified. Each of these sentence structures is [VP [NP]] [V [NP]].

This allows analysis based on cfg rules (11
1) Obtain all parse tree candidates based on the rule corresponding to the cfg rule (112). At this time, when there is a plurality of possible destinations of the prepositional phrase or the adverbial phrase, one of them is provisionally set to one of them.

In the dictionary lookup, the dictionary 18 is indexed with respect to the words included in the analysis result list 200, the contents thereof are fetched, and stored in the dictionary lookup buffer 202. In the provisional tree structuring process, a tree structure is created from the priorities or weights in the analysis result list 200 and the structure editing rules 206 included in the analysis rules 36. The created priority value is stored in the tree structure stack (temporary state of the destination) 216. The tree structure is stored in the tree structure data (temporary state of the destination) 218.

For example, in the above English example, the provisional structure [Vt1 [NP1 NP2 PP1a PP2a]] is obtained. This priority is set to, for example, “25”. This provisional relationship structure is shown in FIG. 4A. Also, another provisional structure [Vt2 [NP1 NP2 PP1a PP2a]] is obtained with other priorities, eg, “20”. This provisional relationship structure is shown in FIG. 4B.
Of course, other than these can also be obtained.

Therefore, the solution candidates are limited based on the priority of the rules (113). Specifically, this is realized by rearranging the tree structure stack (temporary state) and the corresponding tree structure data (temporary state) in descending order of priority of the tree structure (temporary state). I have.

Next, for the parse tree thus narrowed down, a parse tree is extracted for each of the possibilities of the destination (11).
Four).

In the destination determination process 208, the priority is corrected according to a predetermined logical condition 210. For example, the priority of the destination described in the co-occurrence information of the dictionary 18 is increased. The opposite of a dictionary entry reduces priority.
Those not in the dictionary can be left almost unchanged.

The degree of increase or decrease of the priority is preferably determined by a simulation or the like. As an example, the destination node described in the co-occurrence information is +15, if it is related to the parent node of its own node, it is +5, and its own older sibling (sister) of its own node or its descendant. In the case of +8, in the case of other siblings, +2, in the case of parent siblings and their descendants, there is no change, and so on.

In another example, if the relationship is described in a dictionary, +
10, which is a preposition that is easily related to verbs, such as +8 for verbs, -1 for nouns, and the closest one that has more than one possible destination When it is engaged, +5 is applied.

First, one sentence is extracted sequentially from the top of the tree structure data (temporary state of the destination) 218. Next, the co-occurrence information in the sentence is checked against the word description in the dictionary lookup buffer 202. The tree structure and priority are determined according to the contents. At this time, since a plurality of occurrences may occur, a priority table is used. The determined priority is a tree structure stack (dependency determination state) 212
, And the tree structure is tree structure data (decision destination state) 214
To store.

In this embodiment, the range of the change of the destination is indicated by a dotted line 23 in FIG.
The range indicated by 0. That is, in the figure, each circle indicates a node of the syntax tree, and the co-occurrence relationship of each node is indicated by a solid line. Assuming that the own node is i, the range in which the destination can be changed is: 1) the parent node e of the own node i and the ancestor node
b) a, etc. 2) siblings of parent node of own node i that are older than parent node c, d, etc., and descendant nodes n, o, etc. 3) sibling of own node i There are g, h older than the self node and its descendant nodes l, m.

In this way, after extracting all the parse trees defining the possible destinations (114), a more reliable analytic tree is obtained from all the analytic trees obtained in the processing 114 by taking into account the way of the involvement and the like ( 115). Specifically, this is realized by rearranging the tree structure stack (dependency determination state) and the corresponding tree structure data (dependency determination state) in descending order of the priority of the tree structure stack (dependency determination state). I have.

In the above English example, for example, a parse tree [Vt1 [NP1 NP2 [PP1]] PP2] is obtained with a priority of “48”. This is shown in FIG. 6A. This means that in the English example above, the preposition phrase “in the park” relates to the noun phrase “a man”, and the preposition phrase “with a telescope.” Relates to the verb “saw” meaning “saw”. Means

Also, for example, a parse tree [Vt1 [NP1 NP2] PP1 PP2] with priority "45" and a parse tree [Vt2 [NP1 NP2 [PP1]] PP2] with priority "43", and a priority "41" The analytic tree [Vt1 [NP1 NP2 [PP1 [PP2]]]] is obtained. These are shown in FIGS. 6B, 6C and
Each is shown in the 6D diagram.

In the case of FIG. 6B, the preposition phrases “in the park” and “wi
th a telescope. "is a verb" saw "meaning" saw "
In the case of FIG. 6C, the preposition phrase “in the park” relates to the noun phrase “a man” and the preposition phrase “wi
th a telescope. "means the verb" saw "meaning" saw with a saw. "Similarly, in the case of FIG. 6D, the preposition phrase" with a telescope. "
Mean that the latter relates to the noun phrase “a man”.
Of course, other results can be obtained.

The English sentence data thus parsed is converted to a structure conversion unit 24.
And translated into the structure of the Japanese sentence.
Then, a translation is generated for each node included in the translation. The processing of nodes in the structure tree is top-down, left-
Perform with to-right. The generated translation is subjected to post-processing in the post-editing unit 30, is visually displayed on the operation display unit 40, and is printed out as a Japanese sentence 34 in the output unit 32, for example.

As described above, according to the present embodiment, probable solution candidates are narrowed down based on the priority in the English grammar regulation and the like. For a relatively small number of qualified candidates, consider all possible destinations of prepositional phrases and adverbial phrases. As a result, the number of unnecessary solutions is reduced, the efficiency of the analysis is improved, and the analysis result is more reliable.

Effects According to the present invention, candidates for probable solutions are first restricted based on grammar rules, and then all possible destinations of components of a sentence are considered for a limited number of candidates. This minimizes the generation of useless solutions and improves the efficiency and accuracy of the analysis.

[Brief description of the drawings]

FIG. 1 is a functional block diagram showing an entire configuration of an embodiment in which a language analyzer according to the present invention is applied to an English-Japanese automatic translator, and FIG. 2 is a syntax for limiting a participant in the embodiment shown in FIG. FIG. 3 is a flow diagram showing an example of an analysis process. FIG. 3 is a functional block diagram summarizing the functions of a syntax analysis process for limiting a participant in the embodiment. FIGS. 4A to 6D are examples of a specific input English sentence. FIG. 8 is an explanatory diagram showing an example of a process of grouping analytic trees. Explanation of Signs of Main Parts 10 Input Unit 16 Morphological Analysis Unit 18 Word Dictionary 20, 22 Syntactic Analysis Unit 24 Structural Conversion Unit 26 Translation Source Generation Unit 32 Output Unit 36 Analysis rules 38 Control unit 40 Operation display unit

Claims (1)

(57) [Claims] 1. A dictionary means for storing dictionary data for analyzing a sentence in a predetermined language, and a morphological analysis is performed by indexing the dictionary means for the input sentence in the predetermined language. Analysis means for performing syntax analysis on the sentence based on the sentence, wherein the analysis means applies a grammar rule to analyze the surface structure of the sentence, and at this time, is given in advance to the grammar rule In consideration of the priority of the candidate of the destination, a candidate of the temporary destination of the component included in the sentence is obtained with a priority, and the candidate of the temporary candidate having the higher priority among the candidates of the temporary destination is obtained. A candidate for a destination is selected, and at least one of information on the selected candidate for the destination and information on how to bend to the candidate for the destination is referred to, and priority correction information included therein is referred to. Based on the selected destination A language analysis apparatus, wherein the priority of a candidate is corrected, and among the selected candidates for a destination, the corrected destination candidate having a higher priority is further narrowed down and selected.