JPH08278973A - Parallel phrase analyzing device and learning data generating device - Google Patents

Abstract

PURPOSE: To provide the learning data generating device which automatically generates knowledge of parallel structure depending upon fields and context. CONSTITUTION: An inputted learning sentence is converted by a morpheme analysis part 1 into a morpheme string. The morpheme string is converted by a paragraph composition part 2 into a paragraph string. A parallel key decision part 3 decides whether or not there are parallel keys in the paragraph string. When there are the parallel keys, a parallel type classification part 4 checks the parallel type of the learning sentence. A parallel element extraction part 5 extracts parallel elements from the learning sentence when the parallel type is suitable to learning. A modification classification part 6 checks the modification type of the learning sentence. A modification relation extraction part 7 extracts modification elements from the learning sentence when the modification type is suitable to the learning.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel phrase analysis device for recognizing a parallel structure in an input sentence by referring to a learning dictionary and a learning data creation device for creating learning data. The parallel phrase analysis device and the learning data creation device exist in the natural language analysis system.

Documents in general circulation contain a considerable number of parallel structures. A natural language analysis system such as a machine translation system or a document revision support system cannot perform correct analysis without correctly recognizing these parallel structures. Incorrect recognition of parallel structures, in most cases,
The meaning cannot be correctly interpreted, which is fatal when viewed as an analysis as a whole.

[0003]

2. Description of the Related Art In a natural language analysis system, the following has been devised and implemented as a mechanism for recognizing parallel structures. 1. A system operator interactively specifies a parallel structure. The system is automatically analyzed to the point where it can be analyzed, and when the ambiguity of the parallel structure occurs, the system operator is made to specify the parallel range. There are things such as putting parentheses around the parallel range. There is also a method in which possible parallel candidates are presented so that the operator can select the correct one. 2. Those that use syntax information. Use only the syntactically valid ones of the parallel candidates. However, since there are cases where there are multiple syntactically valid parallel candidates, this method alone cannot find the correct one. 3. Uses a parallel structure balance. Among the parallel candidates,
The one with the well-balanced parallel elements on both sides of the parallel key is preferentially treated as the correct answer. 4. The one that uses the case information of idioms. When parallel elements have a dependency relation, the validity of this dependency is taken into account for calculation. 5. Those that use semantic attributes. The validity of the parallel element is calculated from the semantic attribute of the parallel element candidate. 6. A combination of the above information.

[0004]

In the conventional parallel structure recognition as described above, since the information used is often fixed, it is difficult to accurately recognize the parallel structure characteristic of a certain field. It was In the field of document processing, for example, the parallel structure characteristic of the field includes “write” and “read”, “create”, “update”, and “save”. Such a field-dependent structure can be recognized to a certain extent from general information, but a device that can be adjusted according to the field and can be surely recognized is required.

Further, as described above, the parallel structure has characteristics in each field, but often has characteristics in each document. That is, a parallel expression used once in a specific document is often used again in the same form in the same document. It is desirable to be able to adjust such context-dependent information as well.

The present invention was created in view of this point, and it is an object of the present invention to provide a learning data creating apparatus for automatically creating knowledge of a field or context-dependent parallel structure. It is another object of the present invention to provide a parallel phrase analysis device that performs parallel structure recognition by referring to the learning data created by the learning data creation device.

[0007]

According to another aspect of the present invention, there is provided a parallel phrase analyzing apparatus, wherein a morpheme analyzing unit for decomposing an input sentence into a morpheme sequence, a bunsetsu synthesizing unit for synthesizing a phrase sequence from the morpheme sequence, a parallel element and a dependency. It is characterized by comprising a learning dictionary for storing elements and a parallel structure recognizing unit for recognizing a parallel phrase existing in an input sentence based on a phrase sequence and contents of the learning dictionary.

In the parallel phrase analyzing device according to claim 2, in the parallel phrase analyzing device according to claim 1, the parallel structure recognizing unit searches the learning dictionary for all parallel combinations that can be created from the bunsetsu sequence, and the search is performed. It is characterized in that a combination with a large number of hits is preferentially output.

According to a third aspect of the present invention, there is provided a learning data automatic creation apparatus in which a morpheme analysis unit for decomposing a learning sentence into morphemes, a bunsetsu synthesizing unit for synthesizing a morpheme string into bunsetsu strings, and a parallel key for finding a parallel key from bunsetsu strings. Judgment part, parallel type classification part for judging parallel type of learning sentence from phrase sequence and parallel key, dependency type classification part for judging dependency type of learning sentence from phrase sequence, parallel type of learning sentence Based on the above, it is determined whether the learning sentence is useful for learning, and if it is useful, the parallel element extraction unit that extracts parallel elements from the learning sentence, and the learning sentence based on the dependency type of the learning sentence It is characterized by comprising a dependency element extraction unit that determines whether or not it is useful and extracts the dependency element from the learning sentence if it is useful.

According to a fourth aspect of the present invention, there is provided the learning data automatic generation apparatus according to the third aspect, in which the parallel element extracting unit has a parallel ambiguous structure in the parallel structure of the learning sentence in the output result of the parallel type classifying unit. The feature is that only the non-existent objects are targeted for parallel element extraction.

According to a fifth aspect of the present invention, there is provided the learning data automatic generation apparatus according to the third aspect, wherein the dependency element extraction unit is related to the parallel structure of the learning sentence in the output result of the dependency type classification unit. The feature is that only the items that have no ambiguity in the receiving are targeted for the receiving element extraction.

According to a sixth aspect of the learning data automatic generation apparatus of the third aspect, in the learning data automatic generation apparatus of the third aspect, the parallel element extraction unit is a fuzzy parallel to the parallel structure of the learning sentence in the output result of the parallel type classification unit. Even if there is a property, parallel elements are extracted for all possible parallel combinations, and only the frequently used parallel elements are registered as learning data.

According to a seventh aspect of the present invention, there is provided the learning data automatic generation apparatus according to the third aspect, wherein the dependency element extraction unit is related to the parallel structure of the learning sentence in the output result of the dependency type classification unit. Even if there is ambiguity in the dependency, the dependency element extraction is performed for all possible parallel combinations,
The feature is that only the dependency elements having a high frequency are registered as learning data.

A context-following parallel phrase analysis device according to claim 8 is
A morphological analysis unit that decomposes an input sentence into a morpheme sequence, a bunsetsu synthesis unit that synthesizes a bunsetsu sequence from a morpheme sequence, a learning dictionary that stores parallel elements and dependency elements, and a parallel key that finds a parallel key in the bunsetsu sequence. A determination unit, a parallel type classification unit that determines the parallel type of the input sentence from the phrase sequence and the parallel key,
A dependency type classification unit that determines a dependency type of an input sentence from a phrase sequence, a parallel element extraction registration unit that extracts a parallel element from the input sentence by referring to a parallel type of the input sentence, and registers the learning element in a learning dictionary, A parallel phrase that exists in the input sentence based on the dependency element extraction / registration unit that extracts the dependency element from the input sentence by referring to the dependency type of the input sentence, and registers it in the learning dictionary And a parallel structure recognition unit that recognizes

The context-following parallel phrase analysis device of claim 9 is
In the context-following parallel phrase analysis device according to claim 8, the parallel structure recognition unit prioritizes the learning data of the context over the learning data obtained by prior learning.

[0016]

The operation of the parallel phrase analysis device of the present invention will be described. In the learning dictionary, for example, parallel elements such as “edit, print”, “create, update, delete”, “analyze, generate”, “divide and combine”, and “create document”, “edit document”, “update document”.
A dependency element called "document printing" is stored.
For example, when a sentence "edit and print" is input, this input sentence is converted into a morpheme sequence "edit""to""print""wa""line""u"".". It A grammatical attribute is added to each morpheme. The grammatical attribute is, for example, that “edit” is a sahen noun.

A phrase sequence is created from this morpheme sequence. The phrase sequence corresponding to the above input sentence is “edit and” phrase type = body (normal) dependency type = body “print” phrase type = body dependency type = use “do.” Clause type = use dependency The type is called x. From the phrase sequence described above, it can be seen that “edit and” has a word attribute, is a parallel key, and the destination of “edit and” is the word.

Since "edit and" is a parallel key, and the parallel partner is a word, and the parallel element "edit, print" is registered in the learning dictionary, the parallel partner of "edit" is "print". It is understood that it is.

The operation of the learning data creating apparatus of the present invention will be described. For example, when a learning sentence “Edit and print” is input, this learning sentence is converted into a morpheme sequence “Edit”, “To”, “Print”, “W”, “Line”, “U”, “.”. To be converted.

A phrase sequence is created from this morpheme sequence. The phrase sequence corresponding to the above-mentioned learning sentence is “edit and” phrase type = body (normal) dependency type = body “print” clause type = body dependency type = use “do.” Clause type = use dependency The type is called x. The parallel key existing in the above clause sequence is a clause called "edit and". The term “editing and” is related to a word, and in this example, the wording “editing” is only “print”.
A parallel element called is extracted.

When a learning sentence "Edit a document." Is input to the learning data creating apparatus, this learning sentence is "document""no""edit""wa""line""u"" . ".

A phrase sequence is created from this morpheme sequence. The phrase sequence corresponding to the above-mentioned learning sentence is “document” phrase type = body dependency type = body “print” phrase type = body dependency type = use “do.” Clause type = use dependency type = × Is to say. In this phrase sequence, since only the phrase “print” is the target of “document” dependency, the dependency element “print document” is extracted from the learning sentence.

The context-following parallel phrase analysis device of the present invention is a combination of the parallel phrase analysis device of the present invention and the learning data creation device. The context-following parallel phrase analysis device of the present invention extracts learning data from an input sentence and uses the extracted learning data to perform parallel phrase analysis of the subsequent input sentence.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing an example of a learning data creating apparatus of the present invention. In the figure, 1 is a morphological analysis unit, 2 is a clause synthesis unit, 3 is a parallel key determination unit, 4 is a parallel type classification unit, and 5 is a parallel type classification unit.
Represents a parallel element extraction unit, 6 represents a dependency type classification unit, and 7 represents a dependency relationship extraction unit.

The morpheme analysis unit 1 receives a learning sentence and decomposes the learning sentence into a morpheme string. The bunsetsu synthesizing unit 2 assembles a morpheme string obtained from the above morpheme analysis result into a bunsetsu.
The phrase referred to here is a phrase based on standard school grammar. Independent words such as nouns and verbs often have adjuncts. The parallel key determination unit 3 determines the key of the phrase from the phrase sequence synthesized by the phrase synthesis unit 2. The parallel key means a phrase having a form of, for example, “edit (noun) and (parallel particle)”.

The parallel type classifying unit 4 classifies the parallel type of the input sentence from the information of the clause and the parallel key obtained so far. The parallel type indicates, for example, whether there is one or a plurality of bunsetsu that can be juxtaposed with a bunsetsu which is a parallel key. The parallel element extraction unit 5 determines whether or not the learning sentence is useful for learning from the parallel type of the learning sentence determined by the parallel type classifying unit 4, and if it is useful, determines from the learning sentence. Take out parallel elements. The retrieved parallel element is written in the parallel element learning dictionary described later.

The dependency type classifying unit 6 determines the dependency type of the learning sentence from the phrase sequence obtained by the phrase synthesizing unit 2. The dependency type indicates, for example, whether there is one or a plurality of bunsetsu that can be the bunsetsu of the bunsetsu source. The dependency relation extraction unit 7 determines whether the learning sentence is suitable for extracting the dependency information from the dependency type determined by the dependency type classification unit 6, and if so, the dependency element. Take out. The extracted dependency element is stored in the dependency element learning dictionary described later.

FIG. 2 is a diagram showing a flow of processing for creating learning data. In step S1, an input sentence is input. In step S2, the input sentence is decomposed into morphemes.
In step S3, the morpheme sequence is assembled into a phrase sequence. In step S4, the parallel type is determined from the phrase sequence. In step S5, it is determined from the determination result whether the input sentence is suitable for learning. If it is suitable for learning, the parallel element is registered in the parallel element learning dictionary in step S6.

In step S7, the dependency type is determined from the clause sequence obtained in step S3. In step S8, it is determined whether the determined dependency type is suitable for learning. If it is suitable for learning, the dependency element is registered in the dependency element learning dictionary in step S9. In step S10, it is determined whether the input sentence is the last sentence, and if it is the last sentence, the process ends. If not the last, the process returns to step S1 to accept the next input sentence.

FIG. 3 is a diagram for explaining the learning data creation. FIG. 3A shows learning data. In FIG. 3B, the learning data is subjected to morphological analysis and decomposed into a morphological sequence. FIG. 3 (c) shows a morpheme string assembled into a clause. In each clause, the type of clause and the possible preceding clause types are shown. FIG. 3 (c) shows the following. "Edit and"
The phrase that says "is a phrase, is a parallel key,
Indicates that the contact person is a word. In addition, the phrase “print” is a phrase and the contact is a verb. In addition, the phrase "do." Is a verb and has no contact.

From FIG. 3C, it is recognized that the phrase "editing" is a parallel key. In this sentence, the only clause that can be in parallel with "editing" is "printing", so there is no ambiguity in parallel, and thus it is possible to reliably identify parallel elements. From this, this sentence becomes
It is determined that the sentence is suitable for parallel element extraction. The data obtained by extracting and accumulating such parallel element data is shown in FIG.
It is a parallel element learning dictionary of (e).

FIG. 3D shows a phrase extracted from the learning sentence "editing a document". This sentence
There can be only "document" as the target of "editing", and there is no ambiguity of dependency. Therefore, this sentence is judged to be suitable for the modification element extraction, and the modification element is extracted. The dependency element learning dictionary shown in FIG. 3E is obtained by accumulating the dependency elements thus extracted.

FIG. 4 is a diagram for explaining the ambiguity of the parallel structure. In the present invention, sentences are classified according to the type of parallel structure, and it is determined whether the input sentence is suitable for learning. In the example below,
Suppose that "a sentence including a parallel structure and a parallel unambiguous sentence" is a sentence suitable for learning. Since the sentences in FIGS. 4 (a) and 4 (b) both have a parallel key called "reading", they can be said to have a parallel structure.

Further, according to FIG. 4 (a), "reading"
The contact person (in this case, the parallel partner) is a word,
After "reading", there is a wording "to file" and a wording "writing", and it is unclear which one is related. In this way, this sentence is determined to be "ambiguous in parallel structure". On the other hand, in FIG. 4 (b),
There is no ambiguity of parallelism because there is only "write" as the destination of "reading" (parallel partner). In this way, the sentence is determined to be "no parallel ambiguity."

FIG. 5 is a diagram for explaining the ambiguity of dependency. In the present invention, sentences are classified by the dependency type, and it is determined whether or not the input sentence is suitable for learning. In the following example, it is assumed that the “sentence without ambiguity of dependency” is a sentence suitable for learning. In Fig. 5 (a), the word phrase "to file" is related to the idiom, but after "to file", there are two word phrases "written" and "read". It cannot be judged whether or not it is relevant without a full-scale analysis. Therefore, this sentence is referred to as "a sentence with ambiguous dependency".
To determine. In FIG. 5 (b), since the reference to “to file” is only “read”, this sentence is determined to be “sentence without ambiguity in dependency”.

The ambiguity of dependency in the parallel structure of learning sentences will be described with an example. Example 1 (without ambiguity): "I went to Tokyo and Osaka" Example 2 (with ambiguity): "I went to the towns of Tokyo and Osaka" In Example 1, "Tokyo" and "Osaka" are in parallel. Therefore, it can be uniquely recognized that "go" goes to both "Tokyo" and "Osaka". In Example 2, it is possible to interpret that “Tokyo” and “Osaka” are in parallel, and that “Tokyo” and “Osaka town” are in parallel. Therefore, it can be interpreted that "I went" affects "Tokyo" and "Osaka" and "I went" affects "Tokyo" and "town". From this, it can be said that the dependency of this example sentence is ambiguous.

The above operation is performed for all learning sentences to accumulate learning data. In the above example, only unambiguous sentences are targeted for learning.
If there is ambiguity, it is possible to develop it into all possible combinations, store it in the learning dictionary, and discard the infrequent ones. For example, in the case of the above-mentioned example 1, all possible combinations are “I went to Tokyo” and “I went to Osaka.” In the case of the above-mentioned example 2, all possible combinations are “ "I went to Tokyo.""I went to a town in Tokyo.""I went to a town in Osaka."

FIG. 6 is a diagram showing an example of the parallel phrase analysis device of the present invention. In the figure, 1 is a morphological analysis unit, 2 is a clause synthesis unit, 8 is a parallel structure recognition unit, 9 is a parallel element learning dictionary,
Reference numerals 10 respectively indicate dependency element learning dictionaries.

The morpheme analysis unit 1 decomposes the input sentence to be analyzed into morphemes. The phrase synthesizing unit 2 collects the above-mentioned morpheme sequence into a phrase sequence. The parallel structure recognizing unit 8 recognizes the parallel structure from the phrase sequence. The parallel structure recognition unit 8 refers to the parallel element learning dictionary 9 and the dependency element learning dictionary 10 for recognition. The parallel element learning dictionary 9 and the dependency element learning dictionary 10 are created by the learning data creating device (see FIG. 1) prior to analysis.

FIG. 7 is a diagram showing the flow of processing for performing parallel phrase analysis using learning data. In step S1, a sentence is input. In step S2, the input sentence is decomposed into morphemes. In step S3, the morpheme string is assembled into a phrase. In step S4, parallel phrase recognition is performed with reference to the learning dictionary created in the learning stage.

FIG. 8 is a diagram for explaining the recognition of the parallel structure. FIG. 8A shows an input sentence to be analyzed. FIG. 8 (b) shows the input sentence decomposed into clauses. FIG. 8 (c) is an example of a parallel combination that can be created from the phrase sequence. The combinations are shown in parentheses in the figure. Here, the learning dictionary is searched for all the sets of parallel elements and the sets of dependency elements obtained from the respective combinations. As a result of the search, the elements existing in the dictionary are arranged on the right side. Of these, the third combination from the top is the one that most closely matches the contents of the learning dictionary, and is therefore output as the parallel phrase recognition result. FIG. 8 (d) shows the output result. It is also possible to perform scoring by a conventional method for each combination, and to select candidates by adding the score of the search result of the learning dictionary to this score.

FIG. 9 is a diagram showing an example of the context-following parallel phrase analysis device of the present invention. In the figure, 1 is a morphological analysis unit, 2 is a clause synthesis unit, 3 is a parallel key determination unit, 4 is a parallel type classification unit, 6 is a dependency type classification unit, 8 is a parallel structure recognition unit,
Reference numeral 9 indicates a parallel element learning dictionary, and 10 indicates a dependency element learning dictionary.

The context-following parallel phrase analysis device of FIG. 9 is a device for performing parallel phrase analysis using a learning dictionary, and is also configured to perform learning of the learning dictionary at the same time. Basically, it has a structure that combines a structure for learning and a structure for recognition.

In the figure, the components up to the parallel key determination unit 3 are the same as those of an ordinary parallel phrase analysis device. The parallel type classification unit 4 and the dependency type classification unit 6 monitor the bunsetsu string from the parallel key determination unit 3 during the parallel structure recognition processing, and when the sentence is suitable for learning, the parallel element learning dictionary is used. 9 or learning data is added to the dependency element learning dictionary 10. The parallel structure recognition unit 8 performs recognition processing with reference to the dynamically changing learning dictionaries 9 and 10. Therefore, from the recognition of the next sentence in which learning has occurred, the learning result is reflected in the parallel structure recognition unit 8.

In the context-following parallel phrase analysis device of FIG. 9, the learning dictionary contains learning data existing before recognition of the current document and learning data extracted during recognition of the current document. However, the weight of the learning data extracted during the recognition of the current document can be made larger than the weight of the learning data existing before the recognition of the current document.

[0046]

As is apparent from the above description, according to the present invention, when performing parallel phrase recognition, it is possible to reliably recognize parallel expressions that are frequently used in the field to be analyzed. Further, the learning data used in this case can be automatically created without any manual work.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a learning data creation device of the present invention.

FIG. 2 is a diagram showing a flow of processing for creating learning data.

FIG. 3 is a diagram illustrating creation of learning data.

FIG. 4 is a diagram illustrating ambiguity of a parallel structure.

FIG. 5 is a diagram illustrating ambiguity of dependency.

FIG. 6 is a diagram showing an example of a parallel phrase analysis device of the present invention.

FIG. 7 is a diagram showing a flow of processing for performing parallel phrase analysis using learning data.

FIG. 8 is a diagram illustrating recognition of a parallel structure.

FIG. 9 is a diagram showing an example of a context-following parallel phrase analysis device of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Morphological analysis unit 2 Phrase synthesis unit 3 Parallel key determination unit 4 Parallel type classification unit 5 Parallel element extraction unit 6 Dependency classification unit 7 Dependency relationship extraction unit 8 Parallel structure recognition unit 9 Parallel element learning dictionary 10 Dependency element learning dictionary

Claims (9)

[Claims] 1. A morphological analysis unit that decomposes an input sentence into a morpheme sequence, a bunsetsu synthesis unit that synthesizes a bunsetsu sequence from a morpheme sequence, a learning dictionary that stores parallel elements and dependency elements, and a bunsetsu sequence and a learning dictionary. A parallel phrase analysis device, comprising: a parallel structure recognition unit that recognizes a parallel phrase existing in an input sentence based on contents. 2. The parallel structure recognizing unit searches the learning dictionary for all parallel combinations that can be created from the bunsetsu sequence, and preferentially outputs a combination having a large number of search hits. 1. A parallel phrase analyzer of 1. 3. A morpheme analysis unit for decomposing a learning sentence into morphemes, a bunsetsu composition unit for synthesizing a morpheme string into bunsetsu strings, a parallel key determination unit for finding a parallel key from bunsetsu strings, and a bunsetsu string and a parallel key. , A parallel type classification unit that determines the parallel type of the learning sentence, a dependency type classification unit that determines the dependency type of the learning sentence from the clause sequence, and the learning sentence is useful for learning based on the parallel type of the learning sentence. If it is useful, the parallel element extraction unit that extracts parallel elements from the learning sentence and if it is useful, determines whether the learning sentence is useful for learning based on the dependency type of the learning sentence. A learning data automatic creation apparatus comprising: a dependency element extraction unit that extracts a dependency element from a learning sentence if useful. 4. The parallel element extraction unit sets only the parallel output of the parallel type classification unit that has no parallel ambiguity in the parallel structure of the learning sentence as the target of the parallel element extraction. Item 3 learning data automatic creation device. 5. The dependency element extraction unit targets the dependency element extraction only from the output results of the dependency type classification unit, which has no dependency ambiguity in the parallel structure of the learning sentence. The learning data automatic creation device according to claim 3. 6. The parallel element extraction unit performs parallel element extraction for all possible parallel combinations even if there is parallel ambiguity in the parallel structure of the learning sentence among the output results of the parallel type classification unit. 4. The learning data automatic creation apparatus according to claim 3, wherein only the parallel elements having a high frequency are registered as learning data. 7. The dependency element extraction unit is dependent on all possible parallel combinations even if there is an ambiguity in the dependency structure of the learning sentence among the output results of the dependency type classification unit. 4. The receiving element extraction is performed, and only the changing element having a high frequency is registered as learning data.
Learning data automatic creation device. 8. A morphological analysis unit for decomposing an input sentence into a morpheme sequence, a bunsetsu synthesis unit for synthesizing a bunsetsu sequence from a morpheme sequence, a learning dictionary for storing parallel elements and dependency elements, and a bunsetsu sequence for parallelization. A parallel key determination unit that finds a key, a parallel type classification unit that determines the parallel type of the input sentence from the clause sequence and the parallel key, and a dependency type classification unit that determines the dependency type of the input sentence from the clause sequence. By referring to the parallel type of the input sentence, the parallel elements are extracted from the input sentence and registered in the learning dictionary, and by referring to the dependency type of the input sentence, the dependency element is extracted from the input sentence. A context-following type characterized by comprising a dependency element extraction / registration unit for registering in a learning dictionary and a parallel structure recognizing unit for recognizing a parallel phrase existing in an input sentence based on a clause sequence and contents of the learning dictionary. Parallel phrase analyzer. 9. The context-following parallel phrase analysis device according to claim 8, wherein the parallel structure recognition unit prioritizes the learning data of context over the learning data obtained by prior learning.