JPH01144162A - System and device for morpheme analysis using key word - Google Patents

Abstract

PURPOSE:To increase the processing speed with the title system and device by realizing the morpheme analysis with high accuracy for the sentences including character strings, etc., which contain the words proper to the field of a natural language sentence whose topic is limited into a specific field and the specific symbols or the numerical value having the meaning proper to such a field that is not easily recognized as a nit to be divided in the normal morpheme analysis. CONSTITUTION:With input of a natural language sentence whose topic is limited into a specific field, the coincidence is checked between one of those keywords stored preliminarily in a key word storing part 1 and a part of a character string of the input natural language sentence via a divided part 2 obtained by a keyboard. If a coincident part is detected, the punctuation marks recognizable in the morpheme analysis are put at the front and the back of said coincident part. While no mark is given when no coincident part is obtained. Then the natural language sentence divided by said punctuation marks and the undivided natural language sentence containing no punctuation mark are sent to a morpheme analyzing part 3 to undergo the morpheme analysis. Thus the morpheme analysis processing speed is increased.

Description

[Detailed description of the invention] The present invention relates to a method and device for morphologically analyzing

[Prior art]

Conventionally, techniques for morphologically analyzing natural D-language sentences include forced segmentation using one-character type cutting, which is described in Nobumasa Takahashi (ed.), "Japanese Information Processing J," written by Hozumi Tanaka (1986), "Syntactic Analysis and Semantic Analysis"; A morphological analysis technique is known that consists of three processing phases: dictionary lookup and connection information acquisition based on the longest match principle, and separation writing based on a connection table check.

[Problem that the invention seeks to solve]

With conventional morphological analysis technology, the topics covered by natural ah sentences targeted for morphological analysis are limited to specific fields, so if a word unique to a specific field appears in a natural language sentence written by one person, the degree of segmentation becomes difficult. There was a problem with the decline.

[Means for solving problems]

The morphological analysis method using the keywords of the present invention is as follows.

Human-powered natural language sentences with topics limited to specific fields.

A dictionary in which keywords of the specific field are registered in advance is referred to, and when a keyword registered in the dictionary appears, a delimiter that can be recognized by morphological analysis is inserted before and after the keyword, and then morphological analysis is performed. It is characterized by

[Effect]

Field-specific words appear in natural D-word sentences in a certain field. For example, when article sales information is targeted for analysis.

In the sentence ``I'm selling the same new computer I bought for 400,000 yen for 230,000 yen,'' r, lr same item'' means ``like new item,'' which is a candy unique to the field of goods sales. With conventional morphological analysis technology, even if it is possible to divide up to ``...Pancon'', it is highly unlikely that the word ``new same product'' is included in the document, so subsequent divisions cannot be guaranteed. . In an application system of natural language processing based on the target field, it is difficult to accurately pick out words unique to a field because words that are unique to a field usually play an important role in determining the meaning of a sentence. is necessary.

Field-specific candies often have unique meanings within a text, and can be extracted without considering dependence on the first or last candies. "New same product" is one of them.

It can be treated preferentially in the analysis method. That is, first, only keywords are detected from the natural language sentence, and delimiters are inserted as marks before and after the detected keywords, and division is determined at that point. For convenience, the delimiter symbol is represented by @ in the above example.

[I'm selling the new PC I bought for 400,000 yen for 230,000 yen.

becomes. Next, while recognizing that @ is a delimiter, and specifically treating @ as a word that can be freely connected before and after it, conventional morphological analysis techniques are applied to analyze the above sentence.

This method prevents candy unique to one field from being erroneously divided. This is particularly effective when characters that are difficult to recognize as units that are normally divided, such as character strings containing special symbols and numerical values, have field-specific meanings. Furthermore, since the character string targeted for morphological analysis becomes substantially shorter, there is also the effect of speeding up the processing. In the example above.

This results in morphological analysis of two short fragments: ``I bought a bread cone for 400,000 yen'' and ``I will sell it for 230,000 yen.''

〔Example〕

FIG. 1 is a schematic diagram showing an embodiment of the system of the present invention. By keyword division part 2.

It is checked whether any of the keywords stored in advance in the keyword storage unit 1 matches a part of the character string of the natural language sentence written manually, and if there is a matching part, it can be recognized by morphological analysis before and after it. A delimiter is inserted. If there is no match, nothing will be inserted. The natural language sentence divided by the delimiter symbols or the natural language sentence as it has been manually written without the delimiter symbols inserted is sent to the morphological analysis section 3, where it is morphologically analyzed using conventional techniques.

FIG. 2 is a schematic diagram showing an embodiment of the apparatus of the present invention. The human natural language sentence is stored in the human natural language sentence memo IJ11. The controller 15 controls the keyword detector 1
2, the keyword detector 12fi detects keywords while referring to the keyword dictionary 20 in which keywords of a specific field are stored in advance and the input natural language sentence memory 11, and each detected keyword The position IIIt in the input natural language sentence is written into the keyword position memory 14. Keyword position memory 14
Each time a keyword is detected in a human-generated natural language sentence, two values are written: the start position of the keyword in the human-generated natural language sentence, and the position of the character next to the end position. There are various implementation methods for the keyword detector 12 and the keyword dictionary 20.
It is sufficient to use a device based on the 68M algorithm introduced in the August 1987 issue of I, Takagi, and Ushijima, "Realizing 5 types of pattern-matching methods using CD word functions." In the example shown above, ``I'm selling the new Pancon product that I bought for 400,000 yen for 230,000 yen,'' the keyword ``
Position 2 of “iFF” and position 1 of “new same product”
is written into the keyword position memory 14. When the keyword detector 12 finishes scanning the contents of the human natural language sentence memory 11, the controller 15 starts the address counter 1.
3, comparator 16, and multiplexer "9" 17 to operate.

The comparator 16 compares the value of the address counter 13 and the value of the keyword position memory 14, and if there is a negative value in the keyword position memory 14 that is equal to the value of the address counter 13, it acts on the multiplexer 17. It outputs the characters stored in the delimiter symbol memory 18 to the morphological analyzer 19, instructs the address counter 13 to wait for one operation and update the address value, and then outputs the characters stored in the keyword position memory 19. Delete the corresponding value from 14. If there is no value in the keyword position memory 14 that is equal to the value in the address counter 13, the comparator 16 acts on the multiplexer 17 to find the value in the human natural language sentence memory 11 that corresponds to the address value held in the address counter 13. output the characters to the morphological analyzer 19.

Instructs the address counter 13 to add 1 to the address value. The operation of the comparator 16 is repeated until the value of the address counter 13 exceeds the address of the last character of the natural language sentence stored in the input natural language sentence memory 11. In the previous example, up to [the computer I bought for 400,000 yen], characters are sent character by character from human natural language text memo IJ 11 to the morphological analyzer 19, but when the address counter 13 points to "new", the delimiter is sent to the morphological analyzer 19. is sent, and the value of the "new" position is deleted from the keyword position memory 14. Next, "new same product" was sent one character at a time.

When "wo" is reached, a delimiter is sent and the value at the position of "wo" is deleted from the keyword position memory 14. “23
I'll sell it for 10,000 yen. ' will be sent as is.

A conventional morphological analyzer 19 can be used.

〔effect〕

When using the morphological analysis method or device using the keywords of the present invention, when morphologically analyzing a natural language sentence with a topic limited to a certain field, it is possible to analyze words specific to that field and 2. has the effect of being able to morphologically analyze sentences that contain character strings mixed with special symbols and numbers that have field-specific meanings that are difficult to recognize as units of division. Furthermore, since the character string to be subjected to morphological analysis may be shortened, there is an effect of speeding up the processing.

[Brief explanation of the drawing]

There is art. l...Keyword storage unit, 2...Keyword division unit, 3...Morphological analysis unit, 1
1... Memory for input natural language sentences, 12...
・Keyword detector. 13...Address counter, 14...
Keyword position memory, 15...Controller, 1
6... Comparator. 17...Multiple tabs, 18...Memo for delimiters! 7.19 Morphological analyzer,
20...Keyword dictionary, 21...Dividing means by keyword. 22... Morphological analysis means, 23... Keyword storage means. Agent: Patent Attorney Nai・)・'Hara Shin Usuran, - Words and sentences

Claims (2)

[Claims] (1) Input a natural language sentence with a topic limited to a specific field,
A dictionary in which keywords of the specific field are registered in advance is referred to, and when a keyword registered in the dictionary appears, a preset delimiter that can be recognized by morphological analysis is created before and after the keyword in the natural language sentence. Insert the symbol and
A morphological analysis method using keywords, characterized in that a morphological analysis is performed on a natural language sentence into which the delimiter is inserted. (2) A keyword storage means that stores keywords of a specific field in advance, and whether or not there are keywords of the specific field stored in the keyword storage means in a natural language sentence inputted with a topic limited to the specific field. keyword dividing means for inserting preset delimiter signals that can be recognized by morphological analysis before and after the keyword detected in the natural language sentence when the corresponding keyword is detected by referring to the natural language sentence; When the delimiter is found during morphological analysis of the natural language sentence into which the delimiter has been inserted by the dividing means, it is recognized that the character string to be processed has ended, and morphological analysis is started anew from the next character. A morphological analysis device using keywords, characterized in that it includes a repeating morphological analysis means.