JPH06289890A - Natural language processor - Google Patents

Abstract

PURPOSE:To make a correct discriminative read of a sentence in idiomatic expression by discriminatively reading differently read words of the same spelling in a KANJI(Chinese character)-KANA(Japanese syllabary) mixed sentence by utilizing an idiomatic expression dictionary. CONSTITUTION:Restriction among independent words such as 'how to read some word when it appears in a sentence' and restrictions among independent words are not narrowed down but as to words read discriminatively according to restrictions including the adjuncts, the restrictions including adjuncts are described in the idiomatic expression dictionary 4, and the restrictions are applied when a word dictionary 3 is retrieved. Further, a 'case wherein other words are inbetween interposed' is taken into consideration and places where other words can be inserted are describe. The idiomatic expression dictionary 4 which describes those restrictions give restrictions to the retrieval result of the word dictionary 3 and only an optimum reading candidate obtained from the word dictionary 3 can be employed as a retrieval result. Further, special grammar, etc., need not use any method to describe the idiomatic expression dictionary 4, so the discription is easy.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to natural language processing such as a natural language analysis device in text-to-speech synthesis or machine translation.

[0002]

2. Description of the Related Art When analyzing a kanji-kana mixed sentence, there are many kanji that have a plurality of readings, so it is necessary to select an appropriate reading from these. Many can select appropriate ones based on part-of-speech and grammatical information, but there are some that cannot be selected by only using such information.

Conventionally, two approaches have been taken to solve this problem. The first method is to register adjacent words together in a dictionary in the form of a compound word. The second method is a method of registering the reading when a plurality of words co-occur in a sentence as a co-occurrence dictionary.

[0004]

The first method is to consider adjacent word strings as one word collectively, and therefore cannot cope with the case where another word is inserted between them. Since a sequence of different parts of speech is treated as one part of speech, a problem may occur when an attempt is made to perform processing such as modification.

Since the above-mentioned second method describes only the relationship between independent words, when a plurality of independent co-occurring independent words appear in a sentence, some kind of selection criterion is required, Determining the optimal selection criteria is difficult.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a natural language processing device capable of correctly distinguishing homomorphic differently read words.

[0007]

A natural language processing apparatus of the present invention is a natural language processing apparatus for distinguishing homomorphic differently read words in a kanji-kana mixed sentence, and is not limited to the restriction due to the co-occurrence relation of independent words. The present invention is characterized by including an idiomatic expression dictionary (for example, idiomatic expression dictionary 4 in FIG. 1) in which constraints including annexes are described, and using the idiomatic expression dictionary, the above-mentioned reading is performed.

In the description of the conventional expression dictionary, it is preferable to place restrictions on the places where other words are inserted.

It is preferable that the idiomatic expression dictionary is implemented in a format that refers to the contents of the word dictionary.

It is preferable that the application of the idiomatic expression constraint by the idiomatic expression dictionary is performed immediately after the word dictionary is searched.

[0011]

In the natural language processing apparatus of the present invention, not only the constraint due to the co-occurrence relation of independent words but also the conventional expression dictionary describing the constraint including the annexed word is used, and the homomorphic reading in the kana-kana mixed sentence is detected. Reading of words is done. Therefore, it becomes possible to distinguish sentences in an idiomatic expression such as "when a word appears in a sentence, the reading of this word is like this".

In the description of the above-mentioned idiom expression, if restrictions are placed on the positions where other words are inserted, correct reading can be performed even if a word is inserted in the idiom.

Further, when the idiom expression dictionary is realized in a format that refers to the contents of the word dictionary, the idiom expression dictionary does not need to have ancillary information such as the attribute of the word like the word dictionary, and the memory consumption is reduced. The amount can be reduced.

Further, when the application of the idiomatic expressional constraint by the idiomatic expression dictionary is performed immediately after the word dictionary is searched, the number of grammatical analysis processes can be reduced.

[0015]

[Example] In natural language processing for inputting a mixed kanji / kana sentence, since the reading and part-of-speech of kanji and words are not generally one-to-one, it is necessary to select the appropriate reading and part-of-speech. That is, in natural language processing, it is important to perform reading distinction (correct recognition of words).

Generally, a word that is difficult to read is
Words that cannot be narrowed down by grammatical restrictions, that is, homomorphic different words, whose readings change under the influence of other words appearing in the sentence (those other words impose restrictions).

The present invention is intended to distinguish different homomorphic words that cannot be distinguished only by the grammatical constraints appearing in a sentence, and has a constraint caused by a plurality of words as an idiomatic expression dictionary and described in the dictionary. The correct reading is performed by applying the constraint described above.

FIG. 1 shows an embodiment of a natural language processing apparatus according to the present invention. This embodiment is an example in which the present invention is applied to a text-to-speech synthesizer. The feature of this embodiment is that
"When a word appears in a sentence, it reads like this"
Although it cannot be narrowed down only by the constraint between independent words and the constraint between independent words, for those that can be read according to the constraint including the attached word, the constraint including the attached word is described in the idiomatic expression dictionary 4 and the constraint Is applied when searching the word dictionary 3. In addition, considering "when another word is inserted in between" described above, it is also characterized in that the place where the insertion of another word is permitted is also described.

Idiomatic expression dictionary 4 describing these constraints
Is to limit the search result of the word dictionary 3,
It is possible to use only the optimum reading candidate as the search result from the word dictionary 3. Further, the description method of the idiomatic expression dictionary 4 does not require the use of a special grammar, so that the description is easy.

A sentence is input from the input unit 1. The dictionary search unit 2 includes the idiomatic expression dictionary 4 in the sentence input from the input unit 1.
Check if there is an idiomatic expression registered in it.

When an idiomatic expression part is detected in the input sentence, the dictionary search section 2 selects the result of the search of the word dictionary 3 based on the constraint described in the idiomatic expression dictionary 4, and selects other than the idiomatic expression. With regard to the part (2), all the results of the word dictionary search are passed to the analysis processing unit 5 in the subsequent stage.

The word dictionary 3 has information such as words, their readings, attributes, and accents. The idiomatic expression dictionary 4 has information on idiomatic expressions, their readings, places where passages of other words are permitted, and punctuation. The analysis processing unit 5 uses the dictionary search unit 2
The result is input as an input, the word is recognized based on restrictions such as grammar, and the result is passed to the phoneme symbol generator 6 in the subsequent stage.

The phonological symbol generator 6 determines the accent, pause time length, etc. to generate a phonological symbol string. The speech synthesis unit 7 generates synthetic speech based on the phoneme symbol string,
The output unit 8 outputs a synthetic sound.

FIG. 2 is a diagram showing a concrete example of the idiom dictionary, showing idioms, their readings, places where other words can be inserted, and punctuation marks. The example in Fig. 2 is an example in which the word "refuse" is converted into a consonant. When "refuse" is converted into a consonant and becomes "refused", the grammatical constraint is "kotowatari". Indistinguishable from "only" (the one that has become a consonant of "cut off"). Therefore, as shown in FIG. 2, the words are co-occurred and registered as an idiomatic expression (a restriction is provided) to distinguish the words.

Since it is considered that a word is inserted between "customer" and "decline", the place where the insertion of another word is permitted is specified. This makes it possible to distinguish even sentences such as "I refused all customers." The segment break is
This is provided to improve the accuracy of narrowing down the search results of the word dictionary 3.

Further, as shown in FIG. 3, not only the constraint by the co-occurrence relation of the independent words but also the constraint including the annex is described in the idiomatic expression dictionary 4, whereby the "promise is different". You can read differently.
However, not all the annexes are used as constraints, and only the exceptional cases are described. In the example of FIG. 3, “promise / mistake” is described as a constraint.

FIG. 4 is an application example of the idiom dictionary for the sentence including the idiom shown in FIG. The example in Figure 4 "He is
The underlined part is the place where the restrictions of the idiomatic expression dictionary are applied. Word dictionary 3 search results (independent words only)
As shown in FIG. 4, he has 3, he has 5, he has 1, he has 3, he has 3, and he has three. When this result is passed to the analysis processing unit 5 and narrowed down from the grammatical constraint, a plurality of candidates remain in the “refused” part. Therefore, the word dictionary search results are narrowed down by the constraints described in the idiomatic expression dictionary 4, passed to the analysis processing unit 5, and narrowed down from the grammatical constraints, so that the correct reading of "he said all the words" was made. It will be possible.

FIG. 5 shows a processing example of the dictionary search unit 2. The dictionary search unit 2 first searches the idiomatic expression dictionary 4 for the search target character example given from the input unit 1 (step S
1) It is determined whether or not an idiomatic expression exists (step S2). If an idiomatic expression is found in the string,
The dictionary search unit 2 determines the validity of the idiomatic expression (step S3). For example, if a Kanji string that includes a Kanji compound word registered as an idiomatic expression is present in the search target character string, or if the idiomatic expression that allows the insertion of other words, the inserted character string is If it is long, it cannot be guaranteed that it is an idiomatic expression. In order to prevent erroneous analysis due to recognizing such a character string as an idiomatic expression, the validity is determined.

When it is determined in step S3 that the idiomatic expression is valid, the dictionary search section 2 searches the word dictionary 3 for the search target character string to obtain word candidates (step S4). For the idiomatic expression part, candidates are narrowed down by reading written in the idiomatic expression dictionary (step S
5) Regarding the parts other than the idiomatic expressions, all candidates are used as the dictionary search results.

When the idiomatic expression is not detected or when it is judged to be invalid by the validity judgment, the dictionary search unit 2 searches the word dictionary 3 for the character string to be searched and acquires it. All word candidates are used as dictionary search results (step S6).

As shown in FIGS. 2 and 3, the constraints described in the idiomatic expression dictionary 4 do not use a particularly difficult notation. Therefore, editing is very easy. Further, as described above, the idiomatic expression dictionary 4 only needs to describe an exceptional case (which requires exceptional analysis processing if the method of the present invention is not used). Since the exceptional processing is completed before the processing is passed to the section 5, the analysis processing section 5 does not need to perform any special processing, and the amount of calculation for the analysis is reduced.

As shown in FIG. 2, the conventional expression dictionary 4 does not include information such as accents necessary for the analysis processing unit 5 and the phonological symbol generation unit 6 in the subsequent stage, and the information is a word. The information described in the dictionary 3 is used. This allows
Since the idiom expression dictionary 4 can be edited easily and the contents of the word dictionary 3 and the idiom expression dictionary 4 are shared, the problem of inconsistency of contents does not occur. When the sharing of the dictionary contents is realized, the sharing of the dictionary in which the words in the idiomatic expression dictionary 4 refer to a specific address in the independent word dictionary requires the reorganization of both dictionaries due to the modification of one dictionary. Since this is required, the embodiment of the present invention adopts a sharing system in which the contents of the word dictionary 3 are referred to by the entry and the reading in the idiomatic expression dictionary 4.

[0033]

According to the natural language processing apparatus of the present invention, not only the constraint due to the co-occurrence relation of independent words but also the constraint including the ancillary words are used to describe the kanji-kana mixed sentence in the sentence. I decided to distinguish different homonyms from each other.
"When a word appears in a sentence, it reads like this"
It is possible to distinguish sentences such as idiomatic expressions. In addition, since multiple words are not treated as one word, exceptional processing at the time of parsing is not necessary, and the description method of the idiom dictionary is not a special format, so it is easy to describe and edit. It is possible to describe an idiomatic expression dictionary without knowledge of such as.

Further, in the description of the above-mentioned idiomatic expression dictionary,
By placing restrictions on where other words are inserted,
Even if a word is inserted in an idiomatic expression, correct reading can be performed.

Further, by implementing the idiomatic expression dictionary in a format that refers to the contents of the word dictionary, the idiomatic expression dictionary does not need to have attached information such as the attribute of the word unlike the word dictionary, and the memory consumption is reduced. It can be reduced.

Further, by applying the idiomatic expression constraint by the idiomatic expression dictionary immediately after searching the word dictionary,
It is possible to reduce the number of times of grammar analysis processing.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a natural language processing apparatus according to the present invention.

FIG. 2 is a diagram showing a specific example of an idiomatic expression dictionary 4 in FIG.

FIG. 3 is a diagram showing a specific example of an idiomatic expression dictionary 4 in which not only constraints based on the co-occurrence relation of independent words but also constraints including attached words are described.

FIG. 4 is a diagram showing an application example of an idiom dictionary in a sentence including the idiom shown in FIG.

FIG. 5 is a flowchart showing a processing example of the dictionary search unit 2.

[Explanation of symbols]

 2 dictionary search unit 3 word dictionary 4 idiomatic expression dictionary

Claims (4)

[Claims] 1. A natural language processing device for distinguishing homomorphic different words in a kanji-kana mixed sentence, wherein an idiomatic expression dictionary describing not only constraints by co-occurrence relations of independent words but also constraints including annexes is provided. A natural language processing device, comprising: using the conventional expression dictionary to perform the above-mentioned reading distinction. 2. The natural language processing apparatus according to claim 1, wherein in the description of the idiom dictionary, a restriction is placed on a place where another word is inserted. 3. The idiom expression dictionary is realized in a format that refers to the contents of a word dictionary.
The described natural language processing device. 4. The natural language processing apparatus according to claim 1, wherein the application of the idiomatic expressional constraint by the idiomatic expression dictionary is performed immediately after the word dictionary is searched.