JPS6391776A - Natural language analyzer - Google Patents

Abstract

(57) [Abstract] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] A. Industrial Application Field The present invention relates to a natural language analysis device suitable for application to machine translation systems, natural language question answering systems, etc. This makes it easy to select the optimal analytic tree for a given problem.

B. Conventional technology B1. When processing natural language using syntactic and semantic analysis computers, the following are essential:
The goal is to obtain the structure of natural language sentences. In order to do this, it is necessary to refer to a dictionary and determine which word belongs to which part of speech, and further to determine which word modifies which word. A parse tree represents the structure of the sentence obtained in this way. Furthermore, text analysis is the operation of obtaining a parse tree from a natural language sentence. What is difficult in sentence analysis is the interpretation of multipart speech words and the interpretation related to the optional case of verbs. Multi-part speech words are words in which one word behaves as multiple parts of speech; for example, in English, many verbs are also used as nouns.

The optional case of a verb is a case that may or may not exist for a given verb. For example, in cases may or may not be present in many cases. In this case, in may modify the immediately preceding noun in another case, that is, it may not be in case, so it is difficult to handle.

Traditionally, syntactic analysis has been used to obtain parse trees.

In this syntactic analysis, a sentence is made up of several phrases (noun phrases, verb phrases, etc.), and each phrase is made up of several words. Grammar rules are created, and the analysis proceeds according to these rules ( rLanguage As A Co
gnitive Process Vol, I 5yn
taxJ, Terry Winograd, Addis
on Wesley Publishing, rParsingNatur
al LanquageJ, Margaret Kin
g, Academic Press Publishing). The advantage of this syntactic analysis is that it is relatively easy to create grammatical rules and the dictionary descriptions are simple. However, a drawback is that multiple parse trees are output when there is ambiguity due to multi-part speech words, arbitrary cases, etc. mentioned above. Syntactic analysis of natural language sentences rarely yields a unique result, and many different analysis results can be obtained even for relatively short sentences.

By the way, even when there is ambiguity in a sentence, humans can take meaning and context into consideration and provide a unique interpretation. Semantic analysis is the application of such considerations to the analysis of natural language sentences, and one of them is Montague grammar ("Prototype of an English-Japanese translation system based on Montague grammar", Information Processing Society Journal Vol1.
23, Na2, pp, 107-115. March 1982). However, this method is cumbersome because it requires very complicated calculations and the meanings of all words must be written in advance.

This invention focuses on the case structure of verbs to enable statistical processing of ambiguity.

B2. Other conventional techniques for text analysis focusing on case structure include "On the parsing process of the natural language understanding system IMAGES-I, J. Yoshitake et al., Journal of the Institute of Electronics and Communication Engineers Vol. 1. J67LD Nα1
0, ρp, IL47-1154.October 1984, etc.

These methods merely check whether a predetermined case of a verb is an essential case for syntactic analysis, and do not use any statistical information.

A system that uses co-occurrence probability and incorporates some statistical elements is "Syntax analysis method for machine translation", Yoshikawa et al., Information Processing, Vol. 26, N (110, PP, 115
7-1164.October 1985, etc. these are,
It has only been used to analyze compound noun phrases, etc., and there are no examples of it being applied to multi-part words, arbitrary case dependencies, etc.

Furthermore, an attempt at syntactic analysis using statistical information includes "An Approach to Probabilistic Language Processing" by Fujisaki, Information Processing Society of Japan, Natural Language Processing Research Group Material 41-6, January 1984. This method attaches probabilities to the grammatical rules themselves, making the creation of the grammatical rules very difficult, and since the meaning is not taken into account at all, it is difficult to expect results that are appropriate for the field. In addition, since it is influenced by the form of the sentence (such as whether it is passive or not), which is a condition other than semantic elements,
There are contradictions such as not giving the same importance to and virtualmachineJ.

C9 Problems to be Solved by the Present Invention This invention has been made in consideration of the above circumstances, and has been made in consideration of the meaning while maintaining the simplicity and high processing speed of the syntactic analysis method. The purpose is to provide a device that outputs a plausible analytic tree.

Means for Solving Problem D6 In order to achieve the above object, the present invention first performs syntactic analysis of an input sentence. Then, when multiple parse trees are obtained, for all parse trees, the plausibility (likelihood) of the modification between words in a sentence is obtained from predetermined statistical information and summed, and the likelihood of each parse tree is calculated. . Then, the analytic tree with the highest likelihood is output as the most semantically certain analytic tree. Note that modification between words is used in a broad sense and includes relationships such as noun-noun, adjective-noun 1 noun-verb, verb-preposition-noun, etc.

In a more specific example, all modifications between words that appear in the learning data sentences are converted into verb-case relationships, and statistics are taken to obtain data for calculating the likelihood of an analysis tree. That is, when a verb is actually included in the modification, the verb-case relationship is statistically processed. When the case structure does not appear in reality, such as in the noun-noun relationship.

The latent case structure is identified by considering the meaning of the sentence, and verbs are supplemented to obtain the verb-case relationship. And add this to the statistics. Note that the data obtained in this manner will be referred to as the first likelihood calculation dictionary.

If the likelihood of the analytic tree is calculated using only the first likelihood calculation dictionary, the search becomes complicated, so in this specific example, the second likelihood calculation dictionary is also used. That is, in order to obtain statistical information on relationships such as noun-nominal, adjective-noun, etc. using the first likelihood calculation dictionary, it is necessary to convert the noun-noun, etc. relationship into a verb-case relationship. It is difficult to perform this operation during a search. If a second likelihood calculation dictionary that can be directly searched based on the noun-noun relationship is created in advance using the statistical information of the first likelihood calculation dictionary, such Ft complexity can be resolved.

Furthermore, in this specific example, the number of nouns is much larger than verbs, adjectives, etc., and it is not easy to check the frequency of appearance of all nouns, so semantic markers are used. The meaning marker is a classification name for classifying each noun into, for example, 24 types, and is defined by characteristics such as whether it is a human being or the name of software. Thereby, a likelihood calculation dictionary can be created without collecting a huge amount of data. At the time of execution, the syntactic analysis unit first performs syntactic analysis of the input sentence while referring to the grammar rules and the syntactic analysis dictionary. When several parse trees are obtained, the parse tree likelihood calculation unit is activated, searches for the modification of the parse tree, and if it is a modification between a verb and a case, searches the first likelihood calculation dictionary and calculates the modification. Get the likelihood. Also, if it is a noun-noun modification, etc., the second
Search the likelihood calculation dictionary. The likelihood of each modification is accumulated to obtain the likelihood of each analytic tree, and the analytic tree with the highest likelihood is output.

E. Example Hereinafter, an example in which the present invention is applied to an English to Japanese machine translation system will be described with reference to the drawings.

FIG. 1 shows a processing system implementing one embodiment. In Figure 1, work station 1 is the host
It is connected to a computer 2 via a line 3. An auxiliary storage device 5 is connected to the host computer 2 via a channel 4.

The machine translation process realized by the processing system shown in Figure 1 is
As shown in the figure, (Sl) English literary ability, (S2) English sentence analysis, (S3) conversion from English parse tree to Japanese parse tree, (S4) Japanese sentence generation, and (S5) Japanese sentence output. It consists of five stages. Stage S2 is a portion directly related to this invention and will be detailed later. Stage S3 is to generate corresponding Japanese books based on the English parse tree obtained in stage S2, using an English-Japanese dictionary and syntactic correspondence rules. In stage S4, the word order is changed and punctuation marks are inserted to make the Japanese sentence more natural.

A work station 1 in FIG. 1 is used for inputting English sentences, outputting Japanese sentences, and editing sentences.

The host computer 2 is used to execute a program that performs the processing in stages 82 to S4 described above. The auxiliary storage device 5 stores various dictionaries. This dictionary will be explained mainly with reference to FIG.

The third time shows the processing of the English sentence analysis stage S2. Here, for convenience of explanation, the Raku processing is expressed as a block.

In FIG. 3, an input English sentence is supplied to a syntactic analysis section 6. The syntactic analysis unit 6 refers to the syntactic analysis dictionary 7 and the grammar rule table 8 to syntactically analyze the input English sentence. The syntactic analysis dictionary 7 has a data structure shown in FIG. 4, for example, in which headwords and their parts of speech are described. The grammar rule table 8 is, for example, as shown in FIG.

Note that N0UN is a noun, NP is a noun phrase, DET is the limiting side, VERV is a verb, vp is a verb phrase, PREP is a preposition,
PP refers to a prepositional phrase, and S refers to a sentence.

Grammar rules are generally called rewriting rules and are expressed in the form AB-+C. This indicates that if the series AB appears, it will be rewritten as C.

The processing of the syntactic analysis unit 6 in FIG. Time flies 1ike an arrow.

Let's take this sentence as an example. When this sentence is input, the syntactic analysis dictionary 7 is first searched, and tiffie=NOLIN, flies=NOUN,
VERB.

1ike=VERB, PREP, an=DET,
arrow=NOUN is obtained. Then, the following four sets are obtained as candidates.

By applying the grammar rules to the two sets in the upper row, we obtain the parse trees shown in FIGS. 6(a) and 6(b), respectively.

Application of the grammar rules to the two sets in the lower row fails midway through, and no parse tree is generated. vP-vP and N P -P RE
This is because it is not allowed to rewrite P.

The reason why two parse trees are obtained in this way is because there is ambiguity in the text, and in subsequent processing, the parse tree with the most semantic meaning is selected.

The parse tree obtained by the syntactic analysis section 6 in FIG. 3 is supplied to the score calculation section 9 and the optimal parse tree discrimination section 10. If there is only one analytic tree, this analytic tree is output as is through the optimal analytic tree determination section 10.

When there are two or more analytic trees, the score calculation unit 9 refers to the first likelihood calculation dictionary 11 and the second likelihood calculation dictionary 12 to calculate the likelihood of each analytic tree. The optimal analytic tree determination unit 1o compares the likelihoods of the analytic trees and outputs the analytic tree with the maximum likelihood as the optimal analytic tree.

That is, the first and second likelihood calculations 11 and 12
Stores the frequency of modifications between words included in learning data consisting of a large number of sentences. The score calculating unit 9 picks up the modifications expressed in each of the analytic trees, extracts the corresponding frequencies from the first and second likelihood dictionaries 11 and 12, totals them, and uses the result as the likelihood of each of the analytic trees. .

The first likelihood calculation dictionary 11 has a relationship between verbs and their cases 2
The frequency with which a word (phrase) appears in the training data is memorized. An example of the data structure of the first likelihood calculation dictionary 11 is shown in FIG. In this example, 5upport is a verb,
V M / S P, user is the nominative (SUB
J), and program and machine are objective cases (OB, T I). The numbers in the right column represent the frequency of modification. For example, the frequency of modification in which 5upport is a verb and VM/Sp is the nominative is 20.

The second likelihood calculation dictionary 12 stores the frequencies of modifications between nouns and between nouns and prepositions plus nouns. An example of the data structure of this dictionary 12 is shown in FIG. In this example, the noun-noun modification controlprogram occurs 17 times. Also progr
The modification between the noun and the preposition plus the noun "am in disk" occurs five times.

The operation of the score calculation unit 9 in FIG.
roty.

Consider this sentence as an example.

As shown in FIGS. 6(a) and 6(b), there are two parse trees that can be realized for this sentence. and parse tree (a)
Two modifications are extracted for . Note that in this example, D
Since ET (article, restrictive side) does not contain any semantic elements, these are excluded from modification. Also PREP
(Preposition) is combined with the following noun to form a modifier, so the verb and prepositional noun are considered to be one set. Since flyS is a verb in this parse tree, the first likelihood calculation dictionary 11 is searched to obtain the frequencies of the two modifications described above. Let us assume that they are 20 and 10, respectively. Then, the likelihood of the analytic tree (a) becomes the cumulative value 3o.

Regarding the parse tree (b), three modifications are extracted. The time(
The lies modification is a noun-noun modification, unlike the parse tree (a). Therefore, for time-flies, the second likelihood calculation dictionary 12 is searched to obtain, for example, 2.

Also, since 1ile is a verb, the first likelihood calculation dictionary 11 is searched for the remaining two modifications, and for example, 1ile is a verb.
．． 10 each. The likelihood of the analytic tree (b) is 22, which is the cumulative sum of the above values.

The optimal analytic tree discriminator 1o in FIG. 3 compares the likelihoods of the realizable analytic trees from the score calculation unit 9 and outputs the analytic tree with the maximum likelihood. In the above example, FIGS. 6(a) and (b)
) has a likelihood of 3° and 22, respectively, so the analytic tree (a) is output.

Next, a method for creating the likelihood calculation dictionaries 11 and 12 will be described.

First, prepare the learning data. This learning data can be obtained by randomly extracting a large number of sentences from the field targeted for sentence analysis (machine translation). These sentences are then analyzed manually to create a complete, unambiguous parse tree. Then, examine the modifications between words in the parse tree. For example, VM/SP runs in virtual mac
hine.

FIG. 9 shows the case of the input sentence.

(a) is an input sentence for learning, and (b) is an analysis of this input sentence. From now on, VM/SP is the nominative of run,
It turns out that in virtual machine is related to run, and that virtual modifies 1achine, resulting in three modifications. Among these, VM/5P-runs is used as is (however, third person singular present tense S, past tense, etc. are changed to their original forms). Also run-in virtual machine
In ne, the main meaning is run-in machine.
Therefore, a virtual-machine, in which machine is considered to be related to run in the in-case, is semantically equivalent to machine is virtual, so the verb be-virtual is created. This is because an adjective can be treated as a term along with the verb be, and the verb be does not play any role in terms of meaning, so there is no problem. This allows mach
The relationship that ine is the nominative of be-virtual is obtained. From the above, a modification like (c) is extracted.

Although they did not appear here, this, it,
Pronouns such as he and them, articles such as an and the, and numerals such as one, two, and three have little influence when expressing semantic relationships, so
It will not be included in the statistics.

Another example is shown in FIG. In this example, ThxS+is
, a do not affect the semantic relationship, so they are not added to the statistics. Also, machine of VM/S
For P, machine @ of VM/SP
is considered to be qualified. Now, in order to make this modification a relationship between the verb and the case, we make a sentence by supplementing the verb. Then.

Considering this field, rVM/SP runs in m
This task, in which the sentence achineJ is created, is performed by humans, but considering the context before and after 1, it is not a very difficult task for humans, and it only needs to be done once during learning, so it is not a burden on humans. Note that there is a difference in the verbs that are supplemented depending on the field. Accuracy can be improved by reflecting this bias and appropriately selecting verbs to be supplemented. The sentence thus obtained, rVM/SP
The modification (c) is obtained by extracting the relationship between the verb and the case from "runs in machine".

Next, the occurrence of each of the modifications in the example of FIG. 9 and the example of FIG. 10 is counted to obtain the next intermediate frequency data.

Then, similar counting processing is performed for all other learning data sentences to obtain final frequency data, and these are stored in the first likelihood calculation dictionary 11. The data structure at this time has already been described with reference to FIG. Similar processing is performed for other modifications and stored in the first likelihood calculation dictionary 11.

In the above aggregation, words that tend to appear in modifications between verbs and their cases and words that tend to appear in modifications between nouns are treated under the same measure of case structure. , it is possible to ensure that both of these are reflected in the statistical information, and it is possible to accurately and uniformly describe the modification of a word, no matter what tendency the word has.

Below is a summary of points to keep in mind when calculating.

The verb 肱に generally does not modify other words, so we will only consider cases where it is modified. However, except for the following cases, ■ When a noun is modified with ing, the modification relationship between a noun and the verb is investigated.

(Example) programming language.

→human programs in language
gehuman-program, programi
Two relations of n language are obtained.

■When a past participle modifies a noun In this case, the noun is usually the object of the verb.

(Example) Boiled egg.

→human boiled egg Human-boil and boil-egg are obtained.

Adjectives Adjectives are treated as verb equivalents by adding be-.

(Example) file is large-+file-
be-1argelarge file -+fil
Since the e-be-1arge creative adverb generally modifies a verb or a verb equivalent, it is treated as its case.

(Example) ) Luman uses the file
quickly.

use-quickly is taken.

Nouns are used in each case of verbs (nominative, objective, and prepositional), but they can also modify nouns in other ways.

(Example) VM/SP usage.

-+Human uses VM/SP.

human-use, use VM/SP is taken.

(Example) machine of VM/SP.

-) VM/SP runs in machine.

VM/5P-run, run in mach
ine is taken.

Other pronouns, articles, auxiliary verbs, etc. have almost no semantic elements and are therefore not included in the statistics.

Next, a method of creating data for the second likelihood calculation dictionary 12 will be described. To create this data, the operations performed on the modifications between nouns and the like when creating the data for the first likelihood calculation dictionary 11 are performed. That is, when creating the data for the first likelihood calculation dictionary 11, the noun-to-noun modifications, etc. were converted to verb-to-path modifications and then tabulated.

This is because both are the same in meaning. Then, for the same reason, in order to obtain the data of the second likelihood calculation dictionary 12, it is only necessary to convert the modification between verb and path into the modification between noun and noun or the modification between noun and preposition and calculate the total. . Since the statistical information of all the modifications of the learning data sentence has already been stored in the first likelihood calculation dictionary 11 as a modification between the verb and the path, the information stored here is used.

For example, since rmachine of VM/SPJ is semantically the same as rVM/5Pruns in machine J, the conversion is performed in this manner. rIIlac
The score for the phrase hine of VM/SPJ is rV
It should be the same as the score for the sentence M/SP runs in machineJ. The score for this sentence is VM
/5P-run for example 2, run -in mac
hine is 2, for example, and the total is 4. As a result, the modification score of the machine-of VM/SP is determined to be 4, and is stored in the second likelihood calculation dictionary 12.

This concludes the description of the embodiment.

In the above-mentioned embodiment, a check mark is taken for each word of the noun, and then the search is performed. However, the number of noun words is extremely large, and it is difficult to collect statistics on all of them from a limited amount of learning sentences. Therefore, it is preferable to classify noun words into groups according to their characteristics. Here, this group name is called a semantic marker. The table shows examples of semantic markers. Using such semantic markers, for example, rmachine of
VM/SPJ is expressed as rLID of LEJ. If you take a tally using semantic markers and then use this to calculate the likelihood of the parse tree, it will be much easier even if there are few training sentences or even if the input sentences contain words that are not in the training sentences. Processing can be performed without any problems. In an example using table semantic markers, highly accurate sentence analysis was possible after learning several hundred sentences.

Table, semantic marker type HM: human, 0pera
tor, user, etc. AM: Represents an animal a
nimal, fly, catLC: Logical location in the computer area, storage, fileLE: Computer software program, VM/SP, mat
rix related DM: document abstract,
chart, chapter ST: status
5tatus, event, modeTH: technology name batch, trapFA:
Characteristics 5ecurity, optio
nA ding=attribute name class, coun
t, 5izeUD: device machine
e, accessorWK: Job name a
activity, action, creationTM
:time time, past, beg
inningPL: place
, bank, companyPN: person's name
5hannonP○: Point or
igin + position, 1ocatio
n○G: Group Ha department
nt, IBMvA: Attribute value deg
ree, columnLP: logical passage p
ath, bypass, bus, networklF:
Information alarm, answer, c
omentPT: part of a thing, body
edge, headML: object ma
tal, oxideSL: Calculator ply supplies ba
ttery, card, packF9 As described in detail, according to the present invention, when there is ambiguity in the output of syntactic analysis, it is possible to remove the ambiguity by referring to statistical data. The effort required to collect statistical data is relatively small, and it can be implemented using only a simple adder during execution, making it easy to determine multi-item areas and eliminate ambiguity in dependencies, which were previously impossible. Furthermore, by preparing statistical data for each field as a package, analysis results that depend on the field can be obtained.

Furthermore, this invention can be applied to any language, and can be applied to any system that receives natural language sentences as input.

[Brief explanation of the drawing]

FIG. 1 shows a computer system that implements an embodiment of the present invention.
Figure 2 is a flowchart showing the overall flow of this embodiment. Figure 3 is a block diagram showing the main parts of this embodiment. Figures 4 to 10 are the main parts of Fig. 3. FIG. 6... Syntactic analysis section, 9... Score calculation section, 11
...Optimal analysis book discriminator. Applicant International Business Machines Corporation Sub-Agent Patent Attorney Sawa 1) Toshio Host λ Dictionary Grammar Rules Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 (a) VM/SP runs in
Virtual maChine (b) (c) Figure 9

Claims (7)

[Claims] (1) a storage means for storing a value representing the likelihood of modification occurring between constituent elements of a natural language sentence, and a syntactic analysis means for parsing the natural language input sentence into one or more possible modes; When two or more syntactic analysis results are obtained, an accumulating means for accumulating the likelihood of modification appearing in each of the syntactic analysis results; and a syntactic analysis result that maximizes the accumulated value. A natural language analysis device characterized by having a discriminating means for discriminating a result from a regular text analysis result. (2) The natural language analysis device according to claim (1), wherein the modification between the constituent elements is related to a case structure that actually or potentially appears between the constituent elements. (3) The storage means includes a first storage means that is accessed for case structures actually appearing in the input sentence, and a second storage means that is accessed for case structures that potentially appear in the input sentence. and a storage means for
2) Natural language analysis device described in section 2). (4) Claim (3) wherein the first storage means is configured so that the content of the case structure, the constituent elements of the verb related to the case structure, and the constituent elements of the case can be accessed as entries. Natural language analysis device described. (5) The nature described in claim (3) or (4), wherein the second storage means is configured such that a pair of components related by a latent case structure can be accessed as an entry. Language analysis device. (6) Claim (1) in which some of the above constituent elements are classified into one or more groups according to their respective semantic features, and each of the groups is treated as one constituent element.
A natural language analysis device according to items (5) to (5) above. (7) A natural language analysis device according to claim (6), in which some of the constituent elements are constituent elements of nouns.