JPH11167575A - Language analysis system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a language analysis system which can accurately analyze the parts of speech of tokens constructing the sentences. SOLUTION: A division means 2 divides the inputted sentence data into tokens by referring to a dictionary means 8 which previously stores many tokens. A part-of-speech acquisition means 4 decides the part of speech of each divided token by referring to the dictionary 8. If a token has plural parts of speech, a part-of-speech selection means 6 selects or decides a single part of speech for the token based on the parts of speech of one or more tokens preceding or following (or both of preceding and following tokens of) the relevant token. Furthermore, the capacity of the means 8 is significantly reduced and the processing speed is increased by means of a table where the verbs and the suffixes of predicate adjectives are defined in common with each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a language analysis apparatus and method which can be used for machine translation and the like. More particularly, it relates to a technique for accurately determining the part of speech for each token of a language.

[0002]

2. Description of the Related Art Conventionally, machine translation for translating a natural language by a computer has been studied and partially put into practical use. When performing machine translation, an original language to be translated is analyzed and then translated into another language. Therefore, linguistic analysis is an important step in performing machine translation, and the accuracy of linguistic analysis is a major factor in determining the accuracy of translation. Linguistic analysis can be used not only for machine translation but also for other language processing in general.

In the conventional linguistic analysis, when one word (token) has two or more parts of speech, it cannot be specified accurately.

For example, Japanese Patent Laying-Open No. 4-305768 discloses a method for solving this problem by selecting a part of speech by a person. However, in this method, a judgment operation by a person is required, and the part of speech cannot be automatically specified.

Japanese Patent Application Laid-Open No. H5-290081 discloses a method in which the degree of use of a part of speech is stored for each target field of a document to improve the accuracy of specifying the part of speech. According to this method, accuracy can be improved to some extent, but there is a problem that dictionary management is difficult. In addition, there is a limit in improving the accuracy.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a language analysis system and a method capable of solving the above-mentioned conventional problems and accurately specifying a part of speech.

[0007]

According to a first aspect of the present invention, there is provided a language analysis system, comprising: a dividing unit for dividing a given language into tokens; a dictionary unit storing a part of speech for the token; If there is more than one part of speech obtained by the part of speech acquisition means for one token, if one or more parts of speech are obtained by the part of speech acquisition means, one or more A part-of-speech selecting means for selecting one part of speech from two or more parts of speech given to the token based on the part of speech of the token.

According to a second aspect of the present invention, there is provided a language analysis system, comprising: a dictionary unit which stores a part of speech for a token; and a division / part of speech acquisition unit which divides a given language into tokens and acquires a part of speech of each token by referring to the dictionary unit. If there is more than one part of speech obtained by the division / part of speech acquisition means for one token, the token is determined based on the part of speech of one or more tokens located before or after or both of the token. A part of speech selecting means for selecting one part of speech from two or more given parts of speech.

According to a third aspect of the present invention, in the case where there is more than one part of speech for the token, the dictionary means may determine the token based on the part of speech of one or more tokens located before or after or both. It is characterized by having a table for selecting a part of speech.

According to a fourth aspect of the present invention, the dictionary means includes a part-of-speech dictionary in which tokens and parts of speech are associated with tokens having parts of speech other than suffixes of verbs and predicative adjectives. As for the suffix of the verb and the suffix of the predicate adjective, it is characterized by having a table for the suffix of the root of each verb or the root of the predicate adjective.

A linguistic analysis system according to claim 5, wherein the table for idioms that treats, as a suffix, separately or integrally with the suffix table, a suffix containing a part-of-speech token that is not originally a verb suffix. It is characterized by having.

A linguistic analysis system according to claim 6, wherein a table for compound verbs that handles, as a suffix, separately or integrally with a table for a suffix, the one containing a verb token that is not originally a suffix of a verb. It is characterized by having.

According to a seventh aspect of the present invention, there is provided a language analysis method for assigning a part of speech to each token of a given language using dictionary means stored in a storage device. Is stored in the storage device as dictionary means, and the part of speech corresponding to each token in the given language is obtained from the dictionary means. If there is a plurality of parts of speech for one token, It is characterized in that the part of speech of the token is narrowed down based on the part of speech of one or a plurality of tokens located on both sides.

In the language analysis method according to the present invention, when there is more than one part of speech for a token, the part of speech of the token is selected based on the part of speech of one or more tokens located before, after, or both. Is characterized in that the part of speech is selected based on the table.

According to a ninth aspect of the present invention, there is provided a language analysis method comprising:
With respect to verb suffixes and predicate adjective suffixes, tokens are divided by a table for suffixes relating to individual verb roots or predicate adjective roots.

The linguistic analysis method according to claim 10, wherein the table for idioms that treats, as a suffix, separately or integrally with the suffix table, the suffix includes a part of speech token that is not a verb suffix. It is characterized by having.

A linguistic analysis method according to claim 11, wherein the table for compound verbs that handles, as a suffix, a suffix that includes a verb token that is not originally a suffix of a verb, separately or integrally with the suffix table. It is characterized by having.

A storage device according to a twelfth aspect substantially integrates a computer-executable program for executing a method of giving a part of speech to each token of a given language using a computer. A computer readable storage device, the method comprising: storing parts of speech for various tokens in a storage device as dictionary means, obtaining a part of speech corresponding to each token in a given language from the dictionary means, When a token has a plurality of parts of speech, the part of speech of the token is narrowed down based on the parts of speech of one or a plurality of tokens located before or after or both of the token.

According to a thirteenth aspect of the present invention, there is provided a linguistic analysis method, wherein a group including at least a verb suffix and a predicate adjective suffix is separated from other groups, and dictionary means for associating with a part of speech. It is characterized by having.

A language analysis method according to claim 14, wherein a group including at least a root of a verb and a root of a predicate adjective;
It is characterized by having dictionary means for storing the parts of speech of other groups as different parts of speech.

Hereinafter, the concept of terms used to describe the present invention will be described.

"Language": written words such as documents and texts,
It is a concept that refers to natural language such as spoken language. Any format such as a character code format, an image format, and a voice format may be used.

"Division means": means for dividing a language into tokens. In the embodiment, step S2 in FIG.
S3 and S4 correspond. In the embodiment of FIG. 4, tokens other than suffixes are divided with reference to the part-of-speech dictionary.
The suffix is divided with reference to the table for the suffix in FIG. However, the dividing means referred to here is a concept that also includes a means for dividing a suffix with reference to a part-of-speech dictionary. In addition, there is a case where division is performed without referring to the part-of-speech dictionary.

"Part-of-speech dictionary": means for storing the relationship between tokens and parts-of-speech, regardless of the storage format (list format, table format, tree posterior format, etc.). In the embodiment, the dictionary in FIG. In the embodiment of FIG.
Although a part-of-speech dictionary is formed for tokens having parts of speech other than suffixes, a part-of-speech dictionary may be formed including suffixes.

"Dictionary means": a dictionary including at least the above part of speech dictionary. In the embodiment, the concept includes the part-of-speech dictionary 8a of FIG. 1, a table 8b for suffixes of verbs, and a table 8d for suffixes of predicate adjectives. In this embodiment, the table 8b for the verb suffix is a suffix table (table D) following the root of the general verb in FIG. 13, the table in FIG. 14, the table in FIG.
, The compound verb table of FIG. 16, and the like. The table 8d for the suffix of the predicate adjective includes the table in FIG. 17 in this embodiment.

"Part of speech acquisition means": Referring to the dictionary means,
This means means for obtaining the part of speech of the token, and corresponds to steps S2, S3 and S4 in FIG. 4 in the embodiment.

"Part of speech selection means": means for determining the part of speech of a token having two or more parts of speech based on the part of speech of one or more tokens before and after. In the embodiment, step S7 of FIG. ) Corresponds. In the embodiment of FIG. 7, the part of speech is determined with reference to the rule table of FIG. 8, etc., but the part of speech of the token is determined based on the part of speech of the preceding and following tokens regardless of the use of the rule table. It is a concept that includes things. Also, the part of speech of the token before and after
This is a concept used for determining the part of speech, and also includes a part that determines the part of speech in consideration of other elements.

[0028]

According to the first, second, third, seventh, eighth and twelfth aspects of the present invention, when there are two or more parts of speech obtained for one token, the token is located before or after the token, or both. Based on the part of speech of one or more tokens, one part of speech is selected from two or more parts of speech given to the token. In other words, since the part of speech of the token is determined based on the relationship with the preceding and following tokens, the part of speech of the token is determined more accurately, and thereby the exact meaning and the degree of the sentence in the sentence are captured. Becomes possible.

According to the fourth and ninth aspects of the present invention, the dictionary means is provided with at least a table relating to verb suffixes and predicate adjective suffixes. Therefore, only the root of a verb or the root of a predicate adjective can be stored in the part-of-speech dictionary, and the suffix can be shared and stored in a table. That is, since the part-of-speech dictionary only needs to store the roots and the table can be shared, not only the capacity of the dictionary means can be significantly reduced, but also the processing speed can be increased.

According to the fifth and tenth aspects of the present invention, separately from or integrally with the suffix table, a suffix for idioms that include a part-of-speech token that is not originally a suffix of a verb as a suffix is used. It is characterized by having a table. Therefore, it is possible to speed up processing of frequently used idiomatic words following the root of the verb.

According to the sixth and eleventh aspects of the present invention, separately from or integrally with the suffix table, a compound verb that handles a verb token that is not originally a verb suffix is treated as a suffix. It is characterized by having a table. Therefore, the speed of processing for compound verbs can be increased.

A linguistic analysis method according to a thirteenth aspect of the present invention is a dictionary analysis method wherein a group including at least a verb suffix and a suffix of a predicate adjective and a group other than the group are classified and associated with a part of speech. It is characterized by having. Therefore, the verb suffix and the predicate adjective suffix can be processed separately from those having other parts of speech, and the analysis can be made more efficient.

In the fourteenth aspect, a group including at least a root of a verb and a root of a predicate adjective;
It is characterized by having dictionary means for storing the parts of speech of other groups as different parts of speech. Therefore, a process can be performed on the verb suffix or the predicate adjective suffix, triggered by the verb root or the predicate adjective root.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of a language analyzer according to the present invention is shown in FIG. Division means 2
Refers to the dictionary means 8 which stores many tokens in advance, and divides the input sentence data into tokens.
The part-of-speech acquisition unit 4 acquires the part of speech with reference to the dictionary unit 8 for each of the divided tokens. In other words, in this embodiment, the dividing unit 2 and the part of speech
The part-of-speech acquisition means 5 is configured. The part of speech selection means 6
Referring to the rule table 7, if there is a plurality of parts of speech acquired for each token, the part of speech of the token is determined to be one based on the parts of speech of the preceding and / or subsequent (or both) tokens. If the part of speech cannot be determined as one, the narrowing down may be performed.

In the embodiment shown in FIG. 1, the dividing means 2 performs the division with reference to the dictionary means 8. However, the tokens may be divided without referring to the dictionary means 8.

FIG. 2 shows a hardware configuration when the language analyzer of FIG. 1 is realized using a CPU. The bus line 10 includes a CPU 12, a hard disk 14, a CRT
16, memory 18, floppy disk drive (FD
D) 20 and a keyboard 22 are connected. The hard disk 14 has a part-of-speech dictionary 8a that associates tokens with parts of speech, a table 8b for suffixes of verbs (a table 8c for compound verbs is combined, and a table of idioms is integrated), A table 8d for suffixes of predicate adjectives is stored. That is, in this embodiment,
The dictionary means is composed of the part-of-speech dictionary 8a, the table 8b for verb suffixes, and the table 8d for predicate adjective suffixes. Further, a rule table 7 for determining the part of speech is also stored. Further, the hard disk 14 stores a program for performing language analysis. This program, via FDD20,
It is taken from the floppy disk 24.
Of course, the data may be taken from another storage device such as a CD-ROM. Alternatively, the program may be downloaded via a communication line.

The text to be analyzed is stored in the floppy disk 26 and read through the FDD 20. Of course, this may also be read from a medium such as a CD-ROM, or a text sentence may be received by communication. Further, a text sentence input from the keyboard 22 may be an analysis target.

The fetched text is analyzed according to a program stored in the hard disk 14. The analysis result is stored in the hard disk 14 and output to the CRT 16, a floppy disk, or a printer (not shown) as necessary. In some cases, the data may be transferred via a communication line.

The part-of-speech dictionary according to this embodiment stores the parts of speech of each token classified as shown in FIG.
The classification in FIG. 3 is further subdivided. For example, the name group A is further classified into general nouns, nouns that are not general nouns, nouns in the list A, pronouns, and so on. Table 1 shows the sub-classification of the classification of FIG.

The part-of-speech dictionary stores the subdivided parts of speech for each token. The hard disk 12 stores the classification hierarchy of FIG. 3 and the classification hierarchy of Table 1 described later. Therefore, if the subdivided part of speech is known, the higher classification can be easily obtained. For example, it can be easily acquired that "general noun" belongs to "noun group A". In addition, you may make it memorize | store the high-order classification together with the fragmented part of speech.

In FIG. 3 (and Table 1), the verb is
Divided into verb roots and verb suffixes, each of them is one part of speech. Similarly, the predicate adjective is divided into the root of the predicate adjective and the suffix of the predicate adjective, each of which is 1
It has two parts of speech. As a result, the dictionary token is fixed because the token is fixed, translation into another language becomes simple and clear, and a more correct translation can be obtained. Further, the parts of speech are broadly divided into groups of verb roots and predicative adjective roots, and other groups. As a result, it is possible to divide the processing into those with the suffix (the root of the verb and the root of the predicate adjective) and those without the suffix.

In this embodiment, the verb suffix and the predicate adjective suffix are not stored in the part-of-speech dictionary. This is because it is preferable to determine these suffixes by using a table, as described later, both in terms of dictionary capacity and in terms of analysis. In addition,
There are suffixes other than verb suffixes and predicate suffixes (such as numeric suffixes). Hereinafter, unless otherwise specified, a suffix refers to a verb suffix and a predicate suffix.

In the part-of-speech dictionary, it is preferable to store tokens in the order of character codes in consideration of the efficiency of retrieval.

Table 1 (computer part of speech) Name group A 1.1 General nouns: tree, car ... 1.2 Non-generic nouns: enough, necessary ... 1.3 Nouns in List A: present, now ... 1.4 Pronouns: this, which ... 1.5 Nouns to represent: Everyone, many ... 1.6 Specific words: degree, about ... 1.7 "etc.": etc ... 1.8 Basic numbers: 1, 2, ... 1.9 Non-text symbols: 3, H ₂ SO ₄ ... 2. 2. Name group B 2.1 Nounized words: things, things ... 3. Name group C 3.1 Counting words: sheets, books ... What is not a name 4.1 What is not a predicate adjective: big, that ... Postposition 5.1 True postposition: inner, middle ... 5.2 Verb postposition: by, in ... 5.3 Intermediate postposition: by ... 6. Adverbs 6.1 Adjective adverbs: relatively ... 6.2 Simplified adverbs: relatively ... 6.3 Adverbs showing degree: a little, almost ... 6.4 Adverbs from list 1: more, rather ... 6.5 Adverb from list C: about, almost ... 6.6 Manner adverb: poorly, well ... Conjunction 7.1 Equivalent conjunctive: and, and 7.2 Subordinative conjunctive: sometimes, 7.3 7.3 Conjunction "From": From 7.4 Conjunction "Not from, not: From" 7.5 conjunction "If" and " 7. "If, if, even 7.6 Conjunctive to introduce apposition: For example, ie ... Invariants 8.1 Nominal invariants 8.1.1 Invariants ":" are: 8.1.2 Invariants ":": 8.2 When used as an indirect object 8.1.1 Invariants "" 8.1.2 Invariant "to": to 8.3 When used as a direct object: 8.4 Intensifier, restricting, or asking an invariant 8.4.1 Base invariant: ha・ ・ 8.5 Emphasizing invariant: If, then ... Quantifier: like, like ... 10.1 Affixes of numbers: Numbers, eyes ... 10.2 Suffixes of verbs and predicate adjectives: 10.3 Suffixes that are not predicate adjectives: target, target ... 10.4 Quasi-suffixes of nouns: no, possible ... 11. Participants 11.1 Punctuation:. 11.2 Comma :, 11.3 Colon :: 11.4 Special participle 12.1 Roots of general verbs 12.1 Roots of general verbs: reading, writing ... 12.2 Roots of verbs that are not general verbs 13. Root of connective: Then ... 14. Root of auxiliary verb: Nina ... FIG. 4 is a flowchart showing a language analysis program stored in the hard disk 14. First, the text sentence stored in the floppy disk 26 is read (step S1). Next, the CPU 12 divides one sentence of the read text sentence into tokens and acquires a part of speech (step S2). At this time, CPU
Reference numeral 12 decomposes a sentence into tokens based on the part-of-speech dictionary stored in the hard disk 14 to acquire a part of speech.

For example, the process of dividing into tokens and the process of acquiring the part of speech will be described by taking the sentence "I am going to school" as an example. First, the CPU 12 searches the part of speech dictionary for the first character “I”. FIG. 5 shows a part of the part of speech dictionary. As is clear from the figure, "I"
Is a part of speech called a general noun.

Further, the CPU 12 searches the part-of-speech dictionary in the same way for the character "I am". Since “I am” is not stored in the part-of-speech dictionary and “I” is not a kanji, the CPU 12 determines that “I” is one token. At the same time, the part of speech of the token "I" is stored in the analysis file as a general noun of the name group A (see FIG. 6).

Next, it is determined whether the acquired part of speech is either a verb or a predicate adjective (step S3). Here, since it is a general noun, the process proceeds to step S5. In step S5, it is determined whether or not parts of speech have been acquired for all tokens of the sentence. Here, still,
Since there is an unacquired token, the process proceeds to step S6, and the process is performed for the next token.

Next, the part of speech dictionary is searched for "wa". The part-of-speech dictionary does not store the part of speech of "ha",
The number of the rule table is stored. This allows
It can be seen that "ha" has two or more parts of speech. Further, the part-of-speech dictionary is searched in the same manner for the character "hagaku". Since “hagaku” is not stored in the part-of-speech dictionary, the CPU 12 determines that “ha” is one token. At the same time, since there are two or more parts of speech of the token "ha" and it is uncertain, the number B (45) of the rule table stored in the part of speech dictionary is stored (see FIG. 6). In the above embodiment, when there are two or more parts of speech, only the number of the rule table is stored in the part of speech dictionary. However, these parts of speech may be stored together.

In the same manner, "school" and "he" are recognized as tokens, and the parts of speech are stored as shown in FIG. Next, "line" is recognized as a token, and the root of a general verb is stored as the part of speech. Here, since the acquired part of speech is the root of a general verb, step S3 branches to step S4. Step S
In 4, we determine the suffix token and
The analysis of the attribute is also performed. As described below, by using a table, it is possible to simultaneously determine the token and analyze the attribute. If such advantages are not pursued, it is also possible to store the suffixes in the part-of-speech dictionary and process them.

Hereinafter, processing for analyzing a suffix using a table for suffixes of general verbs will be described based on this embodiment. In this embodiment, a "table for a suffix following the root of a general verb" (hereinafter referred to as table D) shown in FIG. Is configured.

As described above, when it is determined that the part of speech of “line” is the root of a general verb, the CPU 12 obtains the column symbol “k” of the table D from the “line” column of the part of speech dictionary of FIG. .

Table D is shown in FIG. This table has columns of r, t, m, b, n, k, k ', g, s, and w. Now, the designated column symbol is "k"
Therefore, the column “k” is referred to. Where C
The PU 12 reads the next character “ki”, and reads the column “k”
Search for the corresponding character in. In this case, since “き” exists in the second line, the analysis is performed using the information of the line. In other words, "ki" has the attribute "simultaneous", and this is stored. Further, the number C (2) of the table to be referred next is acquired.

FIG. 14 shows the table C (2). CPU
12 obtains the next character "ma" and determines whether or not "ma" exists in this table. Further, it is determined whether or not “mas” including the following characters exists in the table. In this way, the CPU 12 searches the table C (2) for one that matches as the longest character string. Here, No. 22 “Masu” is selected. in this way,
At the point when a character consecutive to "ki" is found, the attribute "simultaneous" stored earlier is deleted. This is shown in FIG.
The attributes "simultaneous", "clear process", "not in the past", and "instruction" listed in the table for suffixes shown in Fig. 7 apply only when no suffix follows. However, as an exception, the “instruction” in table C (1)
And the table "C (3)", the attribute of "instruction" remains.
The reason for this exception is that the “instructions” in Tables C (1), C
(3) follows not the verb suffix, but the idiom. Therefore, in the case of table C (4), which follows the verb suffix, the "instruction"
Attribute disappears.

In any case, when "masu" is selected, the attributes "process is clear", "not in the past", and "honorific" are acquired. In addition, "Kimasu" becomes one token and can be determined to be a suffix of a general verb.

If the character string corresponding to the table C (2) cannot be found, the process returns to the table of FIG. 13 and another table (here, S (V)) designated as the next table is used. Search for. Further, another table may be specified from the table C (2). For example, when “Takuna” is selected, an instruction is given to further refer to table C (“C” is stored in the item of the next table). In this case, the following words are searched with reference to the tables C (1) to C (n) (where n is the number of tables with the symbol C).

Based on the above analysis, the CPU 12
As shown in A, "kisuru" is stored as a suffix of a general verb. In addition, the attribute “process is clear”, “not in the past”, and “honorific” are stored. The attributes stored in this way can be used, for example, when translating into another language.

In the above example, the suffix of a general verb has been described. However, the suffix of a connective and the suffix of an auxiliary verb are similarly determined using a table for the suffix and the attribute of the suffix is determined. Perform analysis. However, as for the suffix of the conjunction and the suffix of the auxiliary verb, there is no table D like a general verb, and the table C is directly referred to.

Similarly, for the suffix of the predicate adjective, the suffix is determined and the attribute is analyzed using the suffix table 8d. FIG. 17 shows a part of the table 8d for the suffix of the predicate adjective. In the suffix of the predicate adjective, a table D like the suffix of a general verb is used.
And has a table F as shown in FIG.
The table in FIG. 17 (for a suffix of a predicate adjective) may refer to the table C (for a suffix of a verb) or the table D. Conversely, table C (2) (for the verb suffix) in FIG. 14 is replaced by table D or table F (for the predicate adjective suffix).
May be referred to. This is because there are things that can be both suffixes of verbs and suffixes of predicate adjectives.

As described above, in this embodiment, the suffixes are classified and tabulated. As a result, the dictionary capacity can be significantly reduced as compared with the case where suffixes are assigned to individual verbs and predicate adjectives in the part of speech dictionary. In addition, when determining a suffix, quick processing can be performed, and at the same time, its attributes can be analyzed.
You can also look for improbable suffix connections and find errors in the sentence.

Next, the CPU 12 determines the part of speech for the token whose part of speech is unknown (step S7). FIG.
FIG. 2 shows a detailed flowchart of the part of speech determination. First, in step S10, it is determined whether or not two or more parts of speech have been acquired for "I". Here, since the part of speech of "I" has already been determined as a general noun, it is used as a general noun.

Next, after steps S13 and S14, the same processing is performed for the next token "wa". here,
Since two or more parts of speech have been acquired for "ha" (that is, B (45) is stored), step S1 is executed.
Proceed to 1. In step S11, the stored rule table B (45) is referred to.

The rule table B (45) is stored on the hard disk 14. The details are shown in FIG. A plurality of rules (NO.1 to NO.3) are stored in this rule table. First, the rule of No. 1 is read. N
The rule of O.1 indicates that if the token on the left (one before) is either an invariant, a postposition, or an adverb, the token "ha" is the base invariant. . Here, the CPU 12 reads the part of speech of the token “I” on the left side stored in step S2 (see FIG. 6A).
The part of speech of "I" is a general noun, invariant, postposition,
Since it is not an adverb, the rule of NO.1 does not hold.

Similarly, the CPU 12 examines the rules No. 2 and No. 3 and finds a suitable rule. here,
The rule of NO.3 holds, and the part of speech of “ha” is an invariant “
Is determined to be ".

The CPU 12 stores the part of speech determined in this way on the hard disk 14 (step S1).
2). That is, as shown in FIG. 6B, the invariant "" is stored. Further, the CPU 12 determines the applied rule N
“1” is acquired from the weight item of O.3, and this is also stored (see FIG. 6B). Here, the weight is
It is a numerical representation of the degree of certainty of part of speech determination. In this embodiment, "0" indicates lack of information, "1" indicates accuracy, "2" indicates slight accuracy, and "3" indicates inaccuracy. By assigning such weights, it is possible to add accuracy to the processing results based on the weights in various processes after analysis (for example, translation into another language).

In the same manner as above, processing is performed for all tokens (steps S13 and S14). In the example given here, only the token "ha" has two or more parts of speech, so the final stored content is as shown in FIG. 6B.

As described above, the sentence "I go to school." Can be divided into tokens and given a part of speech. Thereafter, other analysis (sentence pattern analysis, etc.) is performed on the basis of the division into tokens and the attachment of parts of speech (step S9).

In the rule table exemplified above, the part of speech is determined with reference to only the token on the left side. However, other rule tables may refer to the token on the right side (one after) or may refer to only the token on the right side. In any case, it is preferable to refer to tokens before and after (which may include one or more tokens apart) to the extent necessary to determine the part of speech of the token.

Next, the above analysis will be described with reference to the sentence "This village near the foot has a beautiful scenery." First, steps S2, S5, and S6 of FIG. 4 are repeatedly executed (S3 and S4 for verbs and predicate adjectives), and as shown in FIG. The stored contents of the attributes of the predicate adjective are omitted.) Next, in step S7, the part of speech is determined for the token whose part of speech is unknown.

First, with regard to the token “ni”, the rule table B (43) is referred to. FIG. 10 shows details of the rule table B (43). First, the rule 1 is examined. Rule 1 is such that the token on the left is “a verb without a predicate after it”. Here, the root of the verb + the suffix of the verb is called a "verb". This indicates that two tokens (the root of the verb) + the suffix of the verb (such a cluster is called a kwaji word) are used as criteria for judgment. That is, there may be a case where a plurality of tokens are used as criteria for determination as in this case. Further, as shown in Rule 1, the rule may be configured in consideration of an element other than the part of speech, that is, “without a predicate”.

In any case, rule 1 does not apply here because the left token is a general noun. As a result, the rule 2 that is applied in all cases that do not correspond to the rule 1 is applied. That is, the part of speech is determined to be the invariant “ni”. The weight is
2 is stored.

In the same manner, "ha" is read as shown in FIG.
With reference to the rule table B (45), the invariant "" is determined to be "". Further, as for “GA”, it is determined that the invariant “” is “” with reference to the rule table B (44) of FIG.

As described above, the sentence "This village near the foot has beautiful scenery" is analyzed. That is, as shown in FIG. 9B, the division into tokens and the part of speech of each token are stored in the hard disk 14 as an analysis file.

Note that when the same part of speech is continuous, it may be inappropriate to determine the part of speech of the token based on the part of speech of the preceding and following tokens. For example, the result of acquiring the part of speech from the part of speech dictionary for the sentence "Always drive the car quickly, reliably and safely" is as shown in FIG. 12A. Here, for "wa", the part of speech is determined with reference to the rule table B (45) in the same manner as described above.

Next, the rule table B (6) is read to determine the part of speech "quick" (see FIG. 18). However, in accordance with this rule table, the part of speech must not be determined based on the part of speech of the token "sure" on the right side (note that "," here is only used to separate tokens and is ignored). However, in the case illustrated, since the part of speech of “certain” has not been determined, it cannot be determined even. However, even if it can be determined,
Incorrect results based on the part-of-speech of “certain” will produce incorrect results.

The operation will be described with reference to FIG. When the tokens α and β of the same part of speech continue, they are regarded as one token, and the continuous tokens α and β of the same part of speech
It seems to be related to token γ other than. That is,
As shown in the figure, each of the tokens α and β is seen to be related to the token γ. Therefore, for token α, table B (6) in FIG.
When applying, the right token must be γ.

As described above, when tokens of the same part of speech (the tokens referring to the same rule table are presumed to be the same) are continuous (the case where commas or equivalent conjunctions are included is also regarded as continuous), Fit to the rule table using the part of speech of the closest token that does not reference the same rule table (but ignores commas and equivalent conjunctions). In the example above, when determining the part of speech for "quick", instead of "particular" or "safe"
The part of speech of "ni" is decided.

Although the part of speech of "ni" has not been determined and is determined by table B (43), the combination of parts of speech that can be taken in tables B (6) and B (43) is The part of speech can be determined.

Similarly, the part of speech of “sure” is determined by the part of speech of “ni”. The part of speech obtained in this way is shown in FIG. 12B. After the part of speech is determined as shown in FIG. 12B, the "car" is converted into one using a predetermined rule (for example, using a rule such that a continuous general noun and an invariant are used as one token). May be treated collectively as a token. Similarly, "driving" and "shiyo" may be collectively treated as one token and treated as a verb. Such a process can be appropriately selected depending on the content of the subsequent analysis.

As described above, in this embodiment, the verb is divided into the root of the verb and the suffix of the verb,
It has two parts of speech. Similarly, the predicate adjective is divided into a root of the predicate adjective and a suffix of the predicate adjective, each of which is one part of speech. As a result, it is possible to specify the suffix by using the table as described above without registering the suffix in the part-of-speech dictionary.

Conventional words such as "may go" are divided into the verb root "line", the suffix "ku" and the idiom "may", and the idiom "may" May be treated as a suffix and included in a table for verb suffixes. This is because idioms have the same function as verb suffixes. FIG. 15 shows an example of a table of suffixes containing idiomatic terms in this way. This facilitates analysis of the idiomatic term following the root of the verb and its suffix. In this table, the symbols “,” indicate OR, and the symbol “*” indicates AN.
D means. However, the character to the right of "*" may not be used. For example, if (A, B) * (C) is written, it indicates that four possible words are AC, BC, A, and B.

The table that treats idioms as suffixes as described above may be integrated with the verb suffix table, or may be provided separately.

Further, for compound verbs, a compound verb table may be provided which treats the verb following the suffix following the root of the verb (the verb ending in the "i" line) as a suffix. In this embodiment, the compound verb table is provided as a table S (V) indicated by a suffix table following the root of a general verb (see FIG. 16). That is, it is provided in the verb suffix table. However, it may be provided separately from this.

When a verb follows a verb (a verb whose suffix next to the root of the verb ends with "i"), the compound verb table S (V) in FIG. 16 is called. Thereby, the processing can be speeded up.

By the way, verbs such as "to" and "to come" have their roots changed. That is, if the concept of the present invention is applied, there is no root. For such a verb, exceptionally, each changing root is registered in the part-of-speech dictionary, and the number of the next table is stored ("come", "come", "come" in FIG. 5). Come, come.)

In each of the above embodiments, the sentence data is received and divided into tokens to determine the part of speech. However, the sentence divided in advance into tokens is received and the part of speech is determined. Only the operation may be performed.

Further, in each of the above embodiments, each function of FIG. 1 is realized by using the CPU, but a part or all of them may be constituted by hardware logic.

[Brief description of the drawings]

FIG. 1 is a diagram showing an overall configuration of a language analysis system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a hardware configuration when the language analysis system of FIG. 1 is realized using a CPU.

FIG. 3 is a diagram showing classification of parts of speech.

FIG. 4 is a flowchart of a processing program for dividing tokens and acquiring parts of speech.

FIG. 5 is a diagram illustrating an example of a part of speech dictionary.

FIG. 6 is a diagram showing the contents of an analysis file.

FIG. 7 is a flowchart illustrating a program of a part of speech selection process.

FIG. 8 is a diagram showing a rule table B (45).

FIG. 9 is a diagram showing the contents of an analysis file.

FIG. 10 is a diagram showing a rule table B (43).

FIG. 11 is a diagram showing a rule table B (44).

FIG. 12 is a diagram showing the contents of an analysis file.

FIG. 13 is a diagram showing a table (table D) for a suffix following the verb root.

FIG. 14 is a diagram showing a table for verb suffixes.

FIG. 15 is a diagram showing a table for a suffix including a part for treating an idiom as a suffix of a verb.

FIG. 16 is a diagram showing a compound verb table.

FIG. 17 shows a table for suffixes of predicate adjectives.

FIG. 18 is a diagram showing a rule table B (6).

FIG. 19 is a diagram for explaining processing when the same part of speech continues.

[Explanation of symbols]

 2 ... division means 4 ... part-of-speech acquisition means 5 ... division / part-of-speech selection means 6 ... part-of-speech selection means 8 ... dictionary means

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] November 9, 1995

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0040

[Correction method] Change

[Correction contents]

The part-of-speech dictionary stores the subdivided parts of speech for each token. The hard disk 12 stores the classification hierarchy of FIG. 3 and the classification hierarchy of Table 1 described later. Therefore, if the subdivided part of speech is known, the higher classification can be easily obtained. For example, it can be easily acquired that “general noun” belongs to “ name group A”. In addition, you may make it memorize | store the high-order classification together with the fragmented part of speech.

[Procedure amendment 2]

[Document name to be amended] Drawing

[Correction target item name] Fig. 5

[Correction method] Change

[Correction contents]

FIG. 5

Claims (14)

[Claims] 1. A dividing means for dividing a given language into tokens, a dictionary means for storing parts of speech for tokens, a part of speech acquiring means for acquiring the parts of speech of each token divided by the dividing means with reference to the dictionary means, If there is more than one part of speech obtained by the part of speech acquisition means for one token, the token is given to the token based on the part of speech of one or more tokens located before or after or both of the token. A language analysis system comprising: a part of speech selecting means for selecting one part of speech from two or more parts of speech. 2. A dictionary means for storing a part of speech for a token. A division / part of speech acquisition means for dividing a given language into tokens and acquiring the part of speech of each token with reference to the dictionary means. If there are two or more parts of speech obtained by the division / part of speech acquisition means, two or more parts of speech given to the token based on the parts of speech of one or more tokens located before, after, or both of the token A language analysis system comprising: a part-of-speech selecting means for selecting one part of speech from the following. 3. The linguistic analysis system according to claim 1, wherein the dictionary means, when there are two or more parts of speech for the token, the part of speech of one or more tokens located before, after, or both. Characterized in that it has a table for selecting the part of speech of the token based on. 4. The linguistic analysis system according to claim 1 or 2, wherein the dictionary means, for a token having a part of speech other than a verb suffix and a predicate adjective suffix, associates the token with the part of speech. It has a dictionary, and has a table for suffixes related to the root of each verb or the root of a predicate adjective with respect to the verb suffix and the predicate adjective suffix. 5. The language analysis system according to claim 4, wherein the table for the suffix is separate or integrated with the table for the suffix.
Characteristically having a table for idioms that treat suffixes that contain parts of speech that are not verb suffixes. 6. The language analysis system according to claim 4, wherein the table for the suffix is separate or integrated.
A compound having a table for a compound verb that handles a verb token that is not originally a verb suffix as a suffix. 7. Using a dictionary means stored in a storage device,
A language analysis method for assigning a part of speech to each token of a given language, wherein the parts of speech of various tokens are stored in a storage device as dictionary means, and a part of speech corresponding to each token of the given language is stored. When a token has plural parts of speech obtained from the dictionary means, the part of speech of the token is narrowed down based on the parts of speech of one or more tokens located before or after or both of the token. A language analysis method characterized by the following. 8. The language analysis method according to claim 7, wherein when there are two or more parts of speech for the token, the part of speech of the token is selected based on the parts of speech of one or more tokens located before, after, or both. A feature of selecting a part of speech based on a table for performing 9. The language analysis method according to claim 8, wherein at least the suffix of the verb and the suffix of the predicate adjective are converted into a token by a table for the suffix of the root of the individual verb or the predicate adjective. Characterized by performing division. 10. The language analysis method according to claim 9, wherein the table for the suffix is separately or integrally provided.
A table characterized by having a table for idioms that treat suffixes that include part-of-speech tokens that are not originally verb suffixes. 11. The language analysis method according to claim 9, wherein the table for the suffix is separately or integrally provided with:
A compound having a table for a compound verb that handles a suffix that includes a verb token that is not originally a suffix of the verb. 12. A computer-readable storage device in which a computer-executable program for executing a method of assigning a part of speech to each token of a given language using a computer is substantially integrated. In the above method, the part of speech for various tokens is stored in a storage device as dictionary means, the part of speech corresponding to each token in a given language is obtained from the dictionary means, and a plurality of parts of speech for one token are obtained. In some cases, the part of speech of the token is narrowed down based on the part of speech of one or more tokens located before or after or both of the token. 13. A language analysis method for assigning a part of speech to each token of a given language using dictionary means stored in an existing device, comprising: a group including at least a suffix of a verb and a suffix of a predicate adjective. And a dictionary means for separately associating with the other groups and associating them with parts of speech. 14. A language analysis method for assigning a part of speech to each token of a given language using dictionary means stored in a storage device, comprising: a group including at least a root of a verb and a root of a predicate adjective; A feature of having dictionary means for storing parts of speech of other groups as different parts of speech.