JPH10198676A - Device and method for japanese morpheme analysis - Google Patents

Abstract

PROBLEM TO BE SOLVED: To cancel the vagueness of KANA description sentence containing much vagueness by displaying candidates and instructing a correct answer out of displayed candidates from a user when vagueness occurs in token division and KANJI conversion. SOLUTION: A KANJI/KANA mixed candidate sentence display part 10 shows structure prepared by a token list preparing part 9 for the user. A user instruction part 11 makes the user instruct the correct answer out of candidates displayed by a token candidate display part 8 and the KANJI/KANA mixed candidate sentence display part 10. A morpheme analysis control part 12 outputs the morpheme information of sentence inputted by an input part 1 while controlling the KANJI/KANA mixed candidate sentence display part 10 and the user instruction part 11. A storage part 14 stores the sentence inputted by the input part 1, the retrieved result of dictionary retrieval part 3, token data divided by a token dividing part 4, data prepared by a KANJI converting part 7 and the token list preparing part 9, the information of correct answer instructed by the user instruction part 11, and the analyzed result of morpheme analysis control part 12.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a Japanese morphological analysis device and a Japanese morphological analysis method for outputting morphological information of a Japanese sentence input as a character string.

[0002]

2. Description of the Related Art When processing Japanese words such as kana-kanji conversion and machine translation of a word processor, it is necessary to first perform morphological analysis. In morphological analysis, a character string is usually divided into morphemes (smallest units having meaning, hereinafter referred to as tokens) while searching a dictionary storing vocabulary information using words as keys (token division). Morphological information (part of speech, inflection, etc.) Morphological analysis includes methods such as the minimum number of clauses method, the longest matching method on the left, and the minimum cost method, and ambiguity is resolved using these methods. However, none of the methods is perfect and can lead to misinterpretations. In the case of kana notation, that is remarkable.

[0003] For example, in the case of machine translation for converting a Japanese input sentence "Kare ga Kurumatsu" into an English sentence, the English sentence obtained from this input sentence is: He waits in a car. 2. I wait unity he comes. There are two possibilities. If this is described in a sentence mixed with kanji and kana, He waits by car.

[0004] 2. Wait until he comes. It can be seen that there are two types of interpretation in kana-kanji conversion. When these two sentences are evaluated by the above three methods, the minimum number of clauses method: the number of clauses is 4 and both are the same. The longest matching method on the left. "He waits by car." Depending on the way of attachment, "Noun + particle> verb + particle" means "wait until he comes.""Noun + particle <verb + particle" means "he waits by car." Regardless of the method used, it depends heavily on empirical rules, and when processing natural languages that can express a wide variety of situations, it is inevitable that misinterpretations will be introduced.

[0005]

In such a conventional method, an English sentence can be uniquely determined if it is written in a mixture of kanji and kana, but in the case of a solid writing, a correct answer is used unless context information before and after is used. Was difficult to derive. At present, the technology of context understanding has not yet reached the level of practical use.

The present invention solves the above problems, and provides a Japanese morphological analysis method and a Japanese morphological analysis method that can obtain a correct interpretation even in a solid sentence having a plurality of interpretations (kanji conversion candidates). The purpose is to provide.

[0007]

According to a first aspect of the present invention, there is provided a Japanese morphological analyzer for inputting a Japanese sentence as a character string, reading a Japanese word, kanji notation, part of speech information, and a morpheme. A dictionary group storing vocabulary information necessary for analysis, a token division unit for dividing a character string input from the input unit into tokens by referring to the dictionary group, and if the divided token is a hiragana, To kanji conversion, token display and kanji conversion when ambiguity arises, display means for displaying candidates, and instructing means for instructing the user to select a correct answer from the displayed candidates. A configuration was provided.

With this configuration, it is possible to realize a Japanese morphological analysis method and a Japanese morphological analysis method that can obtain a correct interpretation even for a solid text having a plurality of interpretations (kanji conversion candidates).

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The invention of claim 1 is an input means for inputting a Japanese sentence as a character string, and a dictionary which stores Japanese word reading, kanji notation, part of speech information, and vocabulary information necessary for morphological analysis. Group, token dividing means for dividing the character string input from the input means into tokens by referring to the dictionary group, and, if the divided token is a hiragana, a kanji conversion means for converting it into kanji, When an ambiguity occurs in token division and kanji conversion, a configuration including display means for displaying the candidate and instructing means for instructing the user to select a correct answer from the displayed candidates includes a lot of ambiguity. It can eliminate the ambiguity of the kana notation sentence.

According to the second aspect of the present invention, a correct connection relationship is provided by a configuration including a connection check means for examining a connection relation before and after a divided token and a means for displaying a kanji conversion candidate to the user in consideration of the connection relation. Only certain kanji conversion candidates can be displayed to the user.

According to a third aspect of the present invention, when displaying the ambiguity to the user, a structure comprising means for displaying a sentence as a candidate after all the morphological analysis of the input sentence is completed, thereby analyzing a sentence composed of a plurality of sentences. Batch processing can be performed.

According to a fourth aspect of the present invention, in the case where the ambiguity is displayed to the user, the ambiguity is generated at the time when the ambiguity is generated by the configuration including means for displaying a candidate in token units when the ambiguity occurs. Sexuality can be solved, and the result can be used for subsequent morphological analysis to perform the analysis efficiently.

According to a fifth aspect of the present invention, a step of inputting a Japanese sentence as a character string, reading a Japanese word, writing a kanji character,
A step of dividing a group of dictionaries storing part-of-speech information and vocabulary information necessary for morphological analysis, a step of dividing a character string input from the input means into tokens with reference to the group of dictionaries, wherein the divided tokens are hiragana If the ambiguity occurs in token division and kanji conversion, a step of displaying the candidate and a step of instructing the user to select a correct answer from the displayed candidates are included. With the configuration, the ambiguity of the kana notation sentence including many ambiguities can be resolved.

According to a sixth aspect of the present invention, there is provided a configuration including a step of examining a connection relation before and after a divided token and a step of displaying a kanji conversion candidate considering the connection relation to a user, so that a kanji character having a correct connection relation is obtained. Only conversion candidates can be displayed to the user.

According to a seventh aspect of the present invention, when displaying ambiguity to a user, a sentence including a plurality of sentences is analyzed by a configuration including a step of displaying a sentence as a candidate after all morphological analysis of the input sentence is completed. In such a case, batch processing can be performed.

According to the invention of claim 8, when the ambiguity is displayed to the user, the method includes a step of displaying a candidate in token units when the ambiguity occurs. Can be solved, and the analysis can be performed efficiently by using the result for subsequent morphological analysis.

(Embodiments) Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram of a Japanese morphological analyzer according to an embodiment of the present invention, FIG. 2 is a circuit block diagram of the same, FIG. 3 is a diagram showing a flow of a sequential type process, and FIG. FIG. 5 is a diagram showing a flow of a search, FIG. 5 is a diagram showing a flow of the batch processing, and FIG.
FIG. 7 is a diagram showing an example of the dictionary data, FIG. 7 is a diagram showing an example of the connection table, and FIG. 8 is a diagram showing a display example of the batch type.

In FIG. 1, reference numeral 1 denotes an input unit for a user to input a solid written sentence (a sentence containing only hiragana characters).
Reference numeral 2 denotes a dictionary group in which vocabulary information is registered using a character string as a key. FIG. 6 shows an example of the dictionary data used here. As shown in FIG. 6, a character string serving as a key is written in hiragana. This dictionary stores kanji notation information and morpheme information for the kanji notation. As morpheme information, “part of speech”, “conjugation type”, “conjugation type”, “connection information”
Is stored, and the kanji notation information and the morpheme information are collectively referred to as vocabulary information. Further, the data next to the key indicates the number of data in the same kana notation. However, in the same notation, it is assumed that the number of data is stored only in the first entry and 0 is stored in the other entries.

Reference numeral 3 denotes a dictionary search unit for searching the dictionary group 2 using a hiragana character string as a key. Reference numeral 4 denotes a token division unit that cuts out tokens using the result of the dictionary search unit 3. Reference numeral 5 denotes a connection table that defines whether adjacent tokens can be connected. FIG. 7 shows an example of the connection table 5.
The connection table 5 has an array structure as shown in FIG. The rows of the array are called trailing information, and the columns are trailing information. The prefix information (subscripts of columns) and the postfix information (subscripts of rows) are registered as connection information in the dictionary.

Reference numeral 6 denotes a connection check unit for checking whether or not the token extracted by the token division unit 4 and the token preceding it can be connected with reference to the connection table 5. From the viewpoint of the connection table 5, if the intersection of the preceding information of the preceding token and the preceding information of the succeeding token is 1, the adjacent token is connectable, and if it is 0, the connection is added. Reference numeral 7 denotes a kanji conversion unit that converts dictionary data of all tokens that can be connected by the connection check unit 6 into kanji notation by referring to the dictionary data. Reference numeral 8 denotes a token candidate display unit that shows the kanji candidates acquired by the kanji conversion unit 7 to the user. Reference numeral 9 refers to dictionary data of all tokens that can be connected by the connection check unit 6, converts them into kanji, and creates a token list for constructing a structure as shown in FIG. 8 (this is called a token list). Department.

Reference numeral 10 denotes a candidate sentence display unit including Kanji and Kana, which shows the structure created by the token list creation unit 9 to the user. Reference numeral 11 denotes a user instruction unit for instructing the user to select a correct answer from the candidates displayed on the token candidate display unit 8 and the candidate sentence display unit 10 mixed with kanji and kana. Reference numeral 12 denotes a token division unit 4, a connection check unit 6, a kanji conversion unit 7, a token candidate display unit 8, a token list creation unit 9, a kanji kana mixed candidate sentence display unit 10, and a user instruction unit 11, and an input unit. A morphological analysis control unit that outputs morphological information of the sentence input in step 1. Reference numeral 13 denotes a control unit that controls the input unit 1 and the morphological analysis control unit 12. Reference numeral 14 denotes a sentence input by the input unit 1, a search result of the dictionary search unit 3, token data divided by the token division unit 4, data created by the kanji conversion unit 7 and the token list creation unit 9, a user instruction unit 11 Is a storage unit for storing the correct answer information specified by the user and the analysis result in the morphological analysis control unit 12.

FIG. 2 is a circuit block diagram of the Japanese morphological analyzer. Reference numeral 21 denotes a keyboard (including a mouse). 22 is a cathode ray tube display (CRT)
It is. 23 is a central processing unit (hereinafter, CPU). 24 is a random access memory (hereinafter, RAM)
It is. Reference numeral 25 denotes a read-only memory (hereinafter, ROM) for storing a control program and the like. The input unit 1 and the user instruction unit 11 use the keyboard 21 to display the token candidate display unit 8 and the kanji kana mixed candidate sentence display unit 10.
The storage unit 14 is realized by the RAM 24 by the CRT 22. The connection table 5 is stored in the ROM 25, and the dictionary group 2 is stored in any of the RAM 24, the ROM 25, and a secondary storage device. The dictionary search unit 3, token division unit 4, connection check unit 6, kanji conversion unit 7, token list creation unit 9, morphological analysis control unit 12, and control unit 13
The U23 is realized by executing a program stored in the ROM 25 while exchanging data with the RAM 24 and the ROM 25.

The operation of the Japanese morphological analyzer of the present embodiment configured as described above will be described below with reference to the flowcharts of FIGS. 3, 4 and 5.

FIG. 3 shows the flow of processing in which every time an ambiguity occurs in token division and kanji conversion, the ambiguity is displayed and the user is instructed to answer correctly. First, in step S1, a Japanese sentence is input from the input unit 1 in units of one sentence. In this example, it is assumed that a sentence (solid writing sentence) of “kana only” is input.

In step S2, a variable pos indicating the position (character number) of the sentence being processed is initialized. The character number of the first character "ka" is 0. In step S3, po
It is checked whether or not s has reached the number of input characters (here, 10). If not, the process proceeds to step S4, and if it has, the morphological analysis processing of one sentence ends.

In step S4, a dictionary is searched. FIG. 4 shows the dictionary search process. First, in step D1, the first data of the dictionary is read into the variable dic. A variable dNum for counting the number of searched dictionary data is set to 0.
Is stored. In step D2, it is checked whether dictionary data exists, and if so, the process proceeds to step D3.
If not, the dictionary search process ends. When ending the dictionary search process, the dictionary search unit 3 returns the searched dictionary data and the number of data (dNum) to the token division unit 4.

In step D3, the length of the header of the dictionary data read into dic is obtained and stored in the variable len. In step D4, a character string of len characters from the character position pos of the input character string is compared with the dictionary heading. Next, in step D5, the result of the comparison in step D4 is checked. If they match, the process proceeds to step D6, and if they do not match, the process proceeds to step D8.

At step D6, the dictionary data matching the area "result" for storing the search result to be passed to the token dividing unit 4 is added. At step D7, the counter dNum of the number of search data is incremented by one. In step D8, the next dictionary data is read into dic, and the process returns to step D2.

In the first dictionary search process, pos is 0 and the dictionary heading is "?", And "?" For one character from character position 0 of the input sentence matches the dictionary heading. Here, the first to third data in the dictionary match. In step S5, based on the result of the dictionary search unit 3, it is checked whether a matching headline has been searched. If it has been searched, the process proceeds to step S6. If not, an analysis error is notified in step S14. The morphological analysis processing ends.

In step S6, a connection check with the preceding token is performed. At the beginning of the sentence, there are parts of speech that can be the beginning of the sentence and those that are not. Now, as a result of a dictionary search, three items were obtained: "ka" (mosquito)-noun "ka"-particle "kare" (he)-noun, but the particle is a part-of-speech that cannot be the beginning of a sentence. Are "mosquito" and "he". A connection check using a connection table will be described with an example where pos is 3. At this time, the previous token is
-Particles, the trailing information of this token is 0. In the dictionary search in step S4, the following five data items were searched. Check whether connection is possible for each.

1. "Kuru" (repeating)-the final form of the verb,
Prefix information: 2 0 rows and 2 columns in the connection table can be connected with 1.

2. "Kuru" (repeating)-Adverb form of verb,
Prefix information: 2 0 rows and 2 columns in the connection table can be connected with 1.

3. "Kuru" (coming)-the final form of the verb,
Prefix information: 2 0 rows and 2 columns in the connection table can be connected with 1.

4. "Kuru" (coming)-adverb form of verb,
Prefix information: 2 0 rows and 2 columns in the connection table can be connected with 1.

5. "Car" (vehicle)-Noun, prefix information: 1 row 0 column 1 of connection table can be connected with 1.

In step S7, it is checked whether or not there are a plurality of connectable ones.
If not, go to step S10. Here, since all the connections are possible, all the five candidates are displayed in step S8, and the user instructs the correct answer in step S9.

In step S10, it is checked whether or not one connection is possible. If there is one connection, the process proceeds to step S11. If there is not one (no one), an analysis error is notified in step S14 and the morphological analysis process is performed. Ends. Step S11
Then, the token instructed by the user or the token that can be connected only in the connection check is stored in the storage unit 14 as the analysis result. In step S12, the succeeding information of the token stored as the analysis result is stored in the variable con for storing the succeeding information of the preceding token.

In step S13, pos is increased by 1, and
It returns to step S3. FIG. 5 shows a flow of processing in which candidates are displayed for each sentence after all sentence analysis is completed, and the user is instructed to answer correctly. Step T in FIG.
Steps 1 to T4 are the same as steps S1 to S4 in FIG.

At step T5, upon receiving the result of the dictionary search, a connection check between the searched token and the preceding token is performed. The connection check method is the same as that in step S6 in FIG. 3, but here, if the preceding token cannot be connected to any of the following tokens (tokens searched in the dictionary), the preceding token is deleted from the analysis result. Is performed. For example, when the pos is 5, the preceding token is "coming" (repeat)-the final form of the verb, the posterior information: 2 "coming" (repeat)-the union form of the verb, the posterior information: 4 "comes" (Coming)-the final form of the verb, the adjunct information: 2 "Kuru" (coming)-the union form of the verb, the adjunct information: 4, and the candidate for the adjunct token is "to"-particle, previous Contact information: one of 3. If you check each connection possibility,
2 rows and 3 columns in the connection table can be connected with 1 and 4 rows and 3 columns can be connected with 0 and cannot be connected, out of the four preceding tokens, the "coming" (repeat) -verb adjunct form, the posterior information: 4 ""Kuru" (coming)-the adverb form of the verb, and the adjunct information: 4, will be deleted from the analysis result.

Next, in step T6, the existence of a connectable token is checked. If there is a connectable token, the process proceeds to step T7. If not, an analysis error is notified in step T13 and the morphological analysis process ends. . In step T7, the token that has become connectable is added to the token list shown in FIG.

In steps T8 to T11, the succeeding information of the token that can be connected is stored in the array con. First, at step T8, 0 is stored in a counter i. Next, in step T9, it is checked whether or not i has exceeded the number of connectable tokens.
If not, the process proceeds to step T12. Step T1
If 0, the succeeding information of the i-th connectable token is stored in the i-th of the con. In step T11, the counter i is set to 1
Increase and return to step T9. In step T12, the counter pos is incremented by 1, and the process returns to step T3. Finally, the analysis result (token list) is displayed in step T14, and the user is instructed on the correct path.

By performing morphological analysis in this manner, a correct interpretation can be obtained even for a solid sentence having a plurality of interpretations (kanji conversion candidates).

Although the description has been limited to a sentence containing only kana, the same method can be applied to a part of the hiragana notation of a sentence mixed with kanji kana.

[0044]

As described above, according to the present invention, a Japanese morphological analysis method and a Japanese morphological analysis capable of obtaining a correct interpretation even for a solid sentence having a plurality of interpretations (kanji conversion candidates). The method can be realized.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a Japanese morphological analyzer according to an embodiment of the present invention.

FIG. 2 is a circuit block diagram of a Japanese morphological analyzer according to one embodiment of the present invention.

FIG. 3 is a flowchart of a sequential processing of the Japanese morphological analyzer according to the embodiment of the present invention;

FIG. 4 is a flowchart of dictionary search performed by the Japanese morphological analyzer according to the embodiment of the present invention;

FIG. 5 is a collective flowchart of the Japanese morphological analyzer according to one embodiment of the present invention;

FIG. 6 is a diagram showing an example of dictionary data of the Japanese morphological analyzer according to the embodiment of the present invention.

FIG. 7 is a diagram showing an example of a connection table of the Japanese morphological analyzer according to the embodiment of the present invention;

FIG. 8 is a diagram showing an example of a batch display of the Japanese morphological analyzer according to the embodiment of the present invention;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Input part 2 Dictionary group 3 Dictionary search part 4 Token division part 5 Connection table 6 Connection check part 7 Kanji conversion part 8 Token candidate display part 9 Token list creation part 10 Kanji kana mixed candidate sentence display part 11 User instruction part 12 Morphological analysis Control unit 13 Control unit 14 Storage unit 21 Keyboard 22 CRT 23 CPU 24 RAM 25 ROM

Claims (8)

[Claims] An input means for inputting a Japanese sentence as a character string, a dictionary group storing reading of Japanese words, kanji notation, part of speech information, and vocabulary information necessary for morphological analysis, and input from the input means Token dividing means for dividing the divided character string into tokens by referring to the dictionary group, kanji converting means for converting the divided tokens into kanji if the divided tokens are hiragana, and ambiguity in token division and kanji conversion. A Japanese morphological analyzer comprising: display means for displaying a candidate when it occurs; and instructing means for instructing a user to select a correct answer from the displayed candidates. 2. The Japanese language according to claim 1, further comprising: a connection check unit for checking a connection relationship before and after the divided token; and a unit for displaying a kanji conversion candidate to the user in consideration of the connection relationship. Morphological analyzer. 3. The Japanese morphological analysis according to claim 1, further comprising means for displaying a sentence as a candidate after the morphological analysis of the input sentence is completed when displaying the ambiguity to the user. apparatus. 4. The Japanese morphological analyzer according to claim 1, further comprising means for displaying a candidate in units of tokens when the ambiguity occurs when displaying the ambiguity to the user. . 5. A step of inputting a Japanese sentence as a character string, a dictionary group storing reading of Japanese words, kanji notation, part-of-speech information, and vocabulary information necessary for morphological analysis, and input from the input means. A step of dividing the character string into tokens by referring to the dictionary group; a step of converting the divided tokens into hiragana if they are hiragana; and a step of dividing the tokens and converting them into kanji. A method for analyzing Japanese morphemes, comprising: displaying a candidate; and having a user instruct a correct answer from the displayed candidates. 6. The Japanese morphological analysis method according to claim 5, further comprising the steps of: examining a connection relationship before and after the divided token; and displaying to the user a kanji conversion candidate in consideration of the connection relationship. . 7. The Japanese morphological analysis method according to claim 5, further comprising the step of displaying a sentence as a candidate after the morphological analysis of the input sentence is completed when displaying the ambiguity to the user. . 8. The Japanese morphological analysis method according to claim 5, further comprising the step of displaying candidates in units of tokens when the ambiguity occurs when displaying the ambiguity to the user.