JPS61134877A - Unregistered word attribute estimating device - Google Patents

Abstract

PURPOSE:To quicken the processing by estimating a part of speech within the range of a limited part of speech when an unregistered word is a foreign word to apply syntax analysis and applying attribute estimate processing over all attributes when the syntax analysis is failed. CONSTITUTION:When a foreign word part of speech estimation section 4c detects at first whether endings of a foreign word are endings of Sa-line conjugation or not and when they are the endings, it is estimated that an Sa-line noun and a syntax analyzer 5 applies syntax analysis. When the endings are not the Sa-line conjugation endings, whether or not the endings are adjective verb endings is detected, ad when the endings are the adjective verb conjugation endings, it is estimated that the word is an adjective and syntax analysis is applied. When the word is not an adjective verb, it is estimated as a noun, and the syntax analysis is applied. If the syntax analysis is failed, the foreign word transferred to an attribute sequential providing section 4b and the processing similar to unregistered words other than foreign words is attained.

Description

[Detailed Description of the Invention] Technical Field The present invention relates to (1) a Japanese language analysis device for a natural language processing system such as a machine translation system or a word processor, and in particular, to perform syntax analysis and system processing even when an unregistered word appears. The present invention relates to an unregistered word attribute estimating device that automatically estimates the attributes of unregistered words in order to continue the process.

Background Art Natural language processing systems such as machine translation systems and word processors are attracting attention.

Natural language processing systems use Japanese language analysis devices that analyze the syntax of Japanese sentences.

When analyzing a given Japanese syntax, the attributes of each Japanese word are generally determined using a word dictionary in which words and their attributes are registered, and the analysis process is performed based on the connections between these attributes and grammatical rules. It is being done. However, in general, not all words that may constitute a sentence are registered in a word dictionary, and unregistered words often appear in a sentence. In this case, since the attribute for that word cannot be determined, syntactic analysis becomes impossible, and processing for that sentence must be interrupted.

Therefore, several methods for automatically estimating the attributes of unregistered words have been proposed. In other words, ■ A method of sequentially performing brute force attribute estimation like the syntactic analysis method J disclosed in Japanese Patent Application Laid-open No. 58-175074; This is a method that uses a syntactic analysis method to infer based on grammatical rules.

However, these methods have a problem in that it takes a long time to estimate because the estimation method is complicated.

Purpose An object of the present invention is to provide an unregistered word attribute estimating device capable of quickly estimating the attributes of an unregistered word in order to solve the problems of the prior art as described above.

Configuration In order to achieve the above objective number, the configuration of the present invention includes a first means capable of estimating the attributes of the unregistered word across all attributes when an unregistered word that is not registered in a word dictionary appears. An unregistered word attribute estimating device for a natural language processing system comprising: a second means for identifying whether the unregistered word is a foreign word; and estimating the part of speech of the foreign word within a limited range of parts of speech. For foreign words that are unregistered words, the following is added to the estimation results using the method in Section 3.
The configuration of the present invention will be explained in detail with reference to the drawings.

FIG. 1 is a block diagram showing a part of a natural language processing system to which an unregistered word attribute estimating device according to an embodiment of the present invention is applied.

In FIG. 1, 1 is a dictionary search device, 2 is a word dictionary, and 3 is a dictionary search device.
'''1'!1 word detection device/4 is the 0 feature of the present invention
1 is an unregistered word attribute estimation device, 5 is a syntax analysis device, and 6 is an unregistered word attribute estimation device.
is a grammar dictionary. The unregistered word attribute estimation device 4 includes a loanword identification unit 4a, an attribute sequential assignment unit 4b, and a loanword part-of-speech estimation unit 4.
c, and a table of conjugated endings 4d.

An outline of the operation of the natural language processing system shown in FIG. 1 will be explained.

The dictionary search device [1 performs morphological analysis on a Japanese sentence input from an input device (not shown) by checking the attributes (part of speech, etc.) of the word using a word dictionary 2.

By the way, not all words of an input Japanese sentence are registered in the word dictionary 2, and it is inevitable that unregistered words that are not registered in the word dictionary 2 will appear. If an unregistered word appears, the attributes of the unregistered word are unknown, so subsequent processing cannot be performed.

Therefore, in this embodiment, the unregistered word detection device 3 assigns attributes to all the words in the morphologically analyzed Japanese sentence, checks whether or not the morphological analysis has been completely performed, and if the morphologically analyzed word has not been completely analyzed. If there are no unregistered words, the word is directly transferred to the parser f5. When an unregistered word is detected, the unregistered word attribute estimating device 4 estimates the attribute of the unregistered word, gives the estimated attribute to the unregistered word, and sends it to the syntactic analysis device 5. (One of them will be explained in detail later). The syntactic analysis device 5 uses a grammar dictionary 6 to perform syntactic analysis on the morphologically analyzed Japanese sentence. The parsed Japanese sentence is then processed according to the purpose of the natural language processing system. For example 1
In the case of a Japanese-English machine translation system, semantic analysis,
Through processing such as Japanese-English conversion (translation) processing, English syntax generation, and English morpheme generation, an English translated sentence corresponding to the input Japanese sentence is obtained. In the case of a word processor, an appropriate conversion candidate is selected from a plurality of conversion candidates for kana-kanji conversion extracted from the word dictionary 2 using a connection test table, etc., and is used as a conversion result.

The following is an unregistered word attribute estimation bag using Figures 1, 2, and 3! The operation of step 3 will be explained.

FIG. 2 is a flowchart showing the operation of the unregistered word attribute estimation device 4, and FIG. 3 is a diagram showing the contents of the conjugated word ending table 3d.

Unregistered word detection bag! The unregistered words detected by 3 are detected by the unregistered word attribute estimation device! The word is sent to the foreign word identification unit 4a in t4, and it is identified whether the unregistered word is a foreign word (201
). Note that whether or not a word is a foreign word can be easily identified by the word coding system. If it is not a foreign word, it is sequentially sent to the attribute assigning unit 4b, and an attribute (for example, a noun) is assigned (202, 203). This attribute sequential assignment section 4
b is.

This is a mechanism that can assign all attributes. Next, the syntax analysis device 5 performs syntax analysis based on the given attributes (
204). If the parsing is successful, the parsing results are stacked (205.20 F). After stacking or if syntax parsing fails, check whether there are any attributes that have not been assigned yet, and if the above attributes still remain, assign the remaining attributes, and then Perform syntax analysis (207, 203). If there are no unassigned attributes remaining, the stacked analysis results are output and the parsing process ends (20B).

If the unregistered word is a foreign word, the foreign word part of speech estimation unit 4c
The attribute of the foreign word (item 1ii1) is estimated using the conjugated word table 4d as shown in FIG. Since most of the parts of speech of loanwords are nouns, pseudonyms, and adjectives, the conjugation table 4d lists the conjugation endings of these three parts of speech. Therefore, if the attributes of a foreign word are estimated using this conjugation word foot table 4d, only three attributes need to be considered, and one
Since there is no need to consider zero or more attributes, the time required for estimation can be shortened.

The foreign word part-of-speech estimating unit 4c first tests whether the ending of the loan word is a s-flexive conjugation ending, and if it is a s-flexive conjugation ending, it is estimated that it is a s-bending noun, and The syntax is analyzed by the syntax analysis device 5 based on the attribute of noun (209
, 210, 214).

If it is not a subgrade conjugation ending, it is further tested to see if it is an adjective conjugation ending, and the adjective conjugation ending r, fF+
68 go''・adjective 1iiIF, estimated at %6 5・adjective
1 The syntax is analyzed by the syntactic analyzer 5 under the attribute of verb (211, 212, 214).If it is not an adjective, it is presumed to be a noun, and the syntax is analyzed by the syntax analyzer 5 under the attribute of noun. Perform syntax analysis (211
, 213).

Next, the success or failure of the parsing is tested, and if it fails, the foreign word is transferred to the sequential attribute assigning unit 4b, and the same processing as for unregistered words other than foreign words is performed (215, 203).

If parsing is successful.

The parsing process then ends.

In this way, the parts of speech of foreign words are usually nouns,
If an unregistered word is a loan word, we first estimate that it is one of these three parts of speech (attributes). , the syntax of the estimation is performed), and when the analysis fails, the unregistered word attribute estimation process is performed by the adjunct sequential adding unit 4b for the first time, thereby speeding up the unregistered word attribute estimation process. It becomes possible.

In this embodiment, the attribute sequential assigning unit 4 is used as a mechanism that can estimate the attributes of unregistered words across all attributes.
b is employed, but it is also possible to employ other attribute estimation mechanisms, such as an attribute estimation mechanism that provides attributes expected from the grammar of syntactic analysis, in place of the attribute sequential assigning unit 4b.

Effects As explained above, according to the unregistered word attribute estimating device of the present invention, it is possible to speed up the unregistered word attribute estimation process.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing an unregistered word attribute estimation device according to an embodiment of the present invention, FIG. 2 is a flowchart showing the operation of FIG. 1, and FIG. 3 is a diagram showing the contents of the usage amount tail table in FIG. 1. It is a diagram. 4: Unregistered word attribute estimation device, 4a: Foreign word identification unit, 4b
; Attribute sequential assigning unit, 4C: Foreign word part of speech estimation unit, 4d: Conjugation word foot table, 5: Syntactic analysis device.

Claims (1)

[Claims] (1) When an unregistered word that is not registered in a word dictionary appears, unregistered word attribute estimation by a natural language processing system including a first means capable of estimating the attributes of the unregistered word across all attributes The device includes a second means for identifying whether the unregistered word is a foreign word, and a third means for estimating the part of speech of the foreign word within a limited range of parts of speech. An unregistered word characterized in that, for a foreign word that is, the attribute is estimated by the first means only when syntactic analysis based on the estimation result by the third means fails. Attribute estimation device.