JPS63216175A - Natural language processor and natural language question and reply device - Google Patents

Abstract

PURPOSE:To realize natural expression for human being by providing a data relating homonym/synonym and case structure dominated by a word as to a declinable word being a major component of the case structure of a sentence. CONSTITUTION:The titled device consists of an input section 1, a sentence analysis section 2, a word dictionary 3, a semantic structure extraction section 4 and a synonym table 5. Then data of homonym/synonym relating to declinable words being especially important in the syntax of a natural language as th data with respect to the homonym/synonym to a natural language processing unit 10 are provided and the surface case dominant by declinable words is provided. Even when the declinable words are similar (identical) but the surface case structure dominating the declinable words differ such as 'A constitutes B' and 'B consists of A', the processing handling the meaning of a sentence to be similar or identical is attained. Thus, the system copes with versatile expression and more natural expression for human being is realized.

Description

[Detailed description of the invention] [Industrial application field] The present invention relates to a question answering system and an information retrieval system.

In technology such as document processing systems, it relates to natural language processing when inputting and outputting using natural language, and in particular deals with meanings such as synonyms and synonyms in words, and the same (or similar) meaning content in sentences. The present invention relates to a natural language processing device that can handle .

[Conventional technology]

Conventionally, syntactic analysis that reveals the case structure of sentences in natural language has been performed based on, for example, the case grammar proposed by Fillmore. The case structure corresponding to (
This method prepares a case frame) and extracts the content of the sentence (for an overview of such analysis processing, see, for example, "Language Engineering" by Makoto Nagao, published by Shokodo, first edition/printed in 1999,
For reference.

In other words, in syntactic analysis, the most basic rules are the parts of speech of the words that make up the sentence, and the dependency relationships between the subject and predicate, object and predicate, etc. It is equipped with a word dictionary that stores data, divides sentences input in natural language into words, and determines the grammatical role of each word in the sentence.

For example, in Japanese natural language processing, words in sentences,
Determine the nominative case, case particle, object case, etc. In the sentence ``Atoms constitute molecules.'', the verb ``comprise'',
The nominative case is "atomic", the case particles are "wa" and "wo",
The objective case is [specify that it is the molecule ko. Alternatively, it analyzes the dependency relationships of a sentence, such as the relationship between a subject and a predicate, and the relationship between an object and a predicate, and determines the meaning and content by determining the grammatical structure of the sentence by finding words that can be subjects, predicates, or objects. .

[Problem that the invention seeks to solve]

However, conventional natural language processing does not deal with the sameness or similarity of meaning between multiple words. Despite this, the two sentences were sometimes treated as different. In other words, if we have the sentence ``Atoms make up a molecule.'' and the sentence ``A molecule consists of atoms,'' even though these two sentences have the same content, they are ("Configure"
and "consisting of") are different, so these λ sentences are regarded as different sentences.

Therefore, if there is no identification function regarding the sameness or similarity of meaning between two sentences, significant restrictions will be imposed on the operation of many systems using natural language, and natural language input etc. There was a problem in that the convenience was lost.

For example, in a natural language question answering system, the data [Atoms make up molecules. Even if you give the sentence ``What does a molecule consist of?'' and then ask the question ``What does a molecule consist of?'', it is difficult to find an answer to this question. In other words, if there is no information regarding the identity or similarity of the two sentences in terms of their general form and the meaning of the predicates, then in the above example, we can write "What do atoms consist of?" or "
What makes up molecules? Even if you were able to answer a question using the same term ``comprise'', such as
It is impossible to answer the question ``What does a molecule consist of?'', which is the same question as ``What makes up a molecule?''. This restriction of having to use the same terms as the sentences given as data falls far short of the purpose of "natural language processing."

[Failure to solve the problem]

The present invention divides a sentence input using natural language into words, and extracts the meaning of the sentence in natural language based on predefined part-of-speech information of each word, dependency relationships, and word synonyms. It is also equipped with a meaning structure extraction means that generates synonymous and similar expressions, so that even when different terms expressing the same (or similar) meaning are used, they can be treated as the same (or similar) meaning content. It is something to do. It is characterized by having data on synonyms and synonyms so that the meaning content of sentences expressed in different ways can be treated as synonyms and similar meanings, and its purpose is to have fewer restrictions on the sentences that are handled. , it is an object of the present invention to provide a natural language processing device that enables semantic interpretation of even freer text expressions. Such a natural language processing device has an input section that inputs sentences in natural language, a sentence analysis section that grammatically analyzes the sentences input to the input section and reveals the grammatical structure of the sentences, a word dictionary necessary for the grammatical analysis of the sentence in the analysis section; a semantic structure extraction section that extracts the semantic content of the input sentence based on the analysis result of the sentence analysis section;
Conventional technology is equipped with a synonym table for handling the semantic content of sentences, and the conventional technology is to collect data on synonyms and synonyms and the case structure dominated by the words, especially for words that are the main factor in the case structure of sentences. The difference is that he was made to possess.

[Action of the invention]

The data on synonyms and synonyms possessed in the present invention includes synonyms and synonyms data on terms with meanings that are particularly important in natural language sentence analysis, including the surface case relationships governing each term. By possessing
Even if the predicates have similar meanings (synonyms) but differ in the surface case structure that the predicates govern, it is possible to treat the meanings of the sentences as similar (or the same). Therefore, if a natural language processing device is applied to a question answering system, it can handle a variety of expressions and realize more natural interaction for humans. A detailed explanation will be given below with reference to the drawings.

〔Example〕

An embodiment of the invention is shown in FIG. / is an input section, 2 is a sentence analysis section, 3 is a word dictionary, ≠ is a semantic structure extraction section, and y is a synonym table. The input section/ is for inputting sentences in natural language, and can be input by key operation, voice recognition,
Natural language sentences are imported into the device through character encoding processing such as character recognition. The sentence analysis section 2 is the input section/
Analyzes the Japanese grammar of sentences input through the system and determines the grammatical structure of the sentences. This includes morphological analysis, which identifies words in sentences and further divides them into words, and syntactic analysis. The syntactic analysis carried out by the sentence analysis unit 2 seeks expressions as surface case structures or as dependency relationships;
Dependency) Particles, objects, etc. are determined, and the latter defines the dependency relationships between words based on the relationships between the subject and predicate, and the conjunction modifier (object) and predicate.゛The word dictionary 3 stores information used for grammatical analysis in the sentence analysis section. An example of the word dictionary 3 is shown in FIG. 2. The contents of the dictionary consist of a word heading, the word's part of speech, and grammatical information necessary for syntactic analysis.

Taking dependency analysis as an example of syntactic analysis, the part of speech (``ai'' is a transitive verb, ``藍'' is
It is sufficient to know the part of speech (noun), and for particles such as ``ga'', ``wa'', and ``wo'', information necessary for dependency analysis is described in addition to the part of speech. In other words, (first nominative) means that the type of word that is allowed before the particle is nominal, and (second predicate) means that the clause to which the particle is attached as an adjunct can relate to a predicate, and furthermore, The grammatical role of that clause in
The content shown in A of (Rack, A) will be the same.

The Semantic Structure Extraction Department extracts the semantic content of sentences based on the analysis results of the Sentence Analysis Department. Here, the results of syntactic analysis are received, and the sameness (or similarity) of meanings in the sentences is processed, such as predicates and subjects.

Generate knowledge data consisting of a set of modifiers.

The synonym table j contains synonyms and synonyms for headwords (words).
Data of words with similar meanings is stored.

Here, the configuration consisting of the input section/, sentence analysis section 2, and word dictionary 3 is conventionally called a parser (the input section/ is not essential as a parser), and is used for natural language sentence analysis processing. The same processing as before is performed. Therefore, in the present invention, as described above, the semantic structure extraction unit extracts the semantic content of a sentence based on the analysis result of the sentence analysis unit 2. Here, the results of syntactic analysis are received, and the sameness (or similarity) of meaning in the sentences is processed.
Generate knowledge data consisting of a set of predicates, subjects, and modifiers. ”.

Next, the flow of processing in the embodiment shown in FIG. 1 will be explained. The natural language sentences captured by the input unit are divided into words in the sentence analysis unit 2 (morphological analysis).
, the grammatical structure of the sentence is analyzed based on Japanese grammar.

As mentioned above, the syntactic analysis carried out by the sentence analysis unit 2 is to analyze the surface case structure or dependency relationship. First, the surface case structure is divided into the predicate, nominative, case ( Section) Analyze grammatical structures such as particles and objective cases.

For example, assume that the input sentence is "Atoms constitute molecules." In this case, the word "consists" and the nominative "atom"
, the object case is ``molecule'', and the case particles are ``wa'' and ``wo'', respectively. Such analysis is performed based on data stored in the word dictionary 3. In the word dictionary 3, attributes as parts of speech are associated with each word used as a headword. For example, if the well-known part of speech is a noun, it will be the nominative or objective case, and if it is a particle, it will be the case particle, modal particle, etc. The nominative case and object case are determined by the bonding relationship between the noun and the particle. In this way, the predicate, nominative, case particle, objective case, etc. can be determined by the part of speech of a word and the bonding relationship between the words. Also, the dependency relationships between words are shown in Figure 3. In other words, for the input sentence "Atoms constitute molecules.", the dependency relationship between words based on the relationship between the subject and predicate is "atomic (ha)" (Main 9 [Co., Ltd.], "
The dependency relationship between words based on the relationship between the conjunctive modifier (object) and the predicate is the ``numerator (wo)''.
” (object) and “comprise” (predicate). In this manner, the grammatical structure shown in FIG. 3 or the sentence relationship as shown in FIG. 3 is output as a syntactic analysis result.

Next, the semantic structure extraction unit≠ receives the syntactic analysis results shown in FIGS. 3 and 3 and generates knowledge data having predicates, subjects, and modifiers as elements. The structure of the knowledge data is briefly explained here (predicate, subject.

This expression defines the meaning of the sentence that naturally results from the relationship between subject and predicate as shown in Figure μ, and the relationship between conjunction modifier and predicate. For example, this is defined as a relationship between atoms and molecules.

Knowledge data such as ``atoms (9 molecules)'' expresses the semantic content ``atoms constitute molecules.''

Iζ Here, the elements of the knowledge data can be handled as variables in order to determine the synonyms and similar meanings of the sentences by making them correspond to the knowledge data. That is, two input sentences "Atoms constitute a molecule." and "A molecule consists of atoms." will be explained as an example.

If the input sentence is "A molecule consists of atoms," the corresponding knowledge data is, for example, (2 molecules consisting of, atoms), and in order to treat these two knowledge data as the same (or similar), Consists of (consists of +A+B) and (.

It is necessary to be able to process that B and A) represent the same (or similar) meaning (here, the alphabets A and B are the meanings of variables, h is an atom, and B is a molecule). . For the semantic structure extraction department Hiroshi, data on synonyms and similar meanings of terms (words) is provided as a synonym tableter, and the knowledge data (consists of 9 atoms) and (
Consists of.

molecules, atoms) can be treated as synonyms.

FIG. 5 shows the structure of the stored data in the synonym table j. The synonym table j is a table in which a headword is associated with other words having M similar meanings. As shown in the example of the input sentence here, in the case of a word that explains the relationship between multiple words (here I have simply shown the case of two) that are connected by the meaning of the word, It is also necessary to simultaneously handle the roles of these multiple words in the sentence (here called surface cases), and in the case of a relationship between two words, the nominative (subject) and object (in Figure 3) object)(
This corresponds to two types of positions such as subject and conjunction modifier (object) in Figure 1, and there are two types of relationships: whether this is the same as or opposite to the role of each word in the headword (surface case). It turns out that there is. The synonym table in Figure 5! So, as a synonym for 2 Enoki-class millet corresponding to the 7 headwords/,
It is classified into 2 and shown. Here, / means the words have the same role (surface route), and 2 means the opposite.

Therefore, based on the semantic structure of the input sentence extracted by the semantic structure extraction unit can also be made equivalent to inputting the expression at the same time.

FIG. 2 is a flowchart showing the process of generating synonymous/similar meaning expressions. Here, we input the knowledge data (comprise, one atom, molecule) resulting from the analysis of the input sentence, first check whether there is a synonym for the term "comprise" by referring to the synonym table j, and then The first synonym "to generate" and its case structure information (1) are extracted, and new knowledge data (corresponding to case structure information (1) in which the second and third terms in the knowledge data have the same order) Generate.Atom 2
molecule) and find the next synonym.

Below, the same operation is repeated, but when extracting the synonym "consisting of" and its case structure information (2),
Corresponding to case structure information (2), among the elements of knowledge data,
New knowledge data with the order of the second and third terms reversed (
(9 molecules, atoms) consisting of In this way, new knowledge data is generated for all the synonyms in the synonym table, and the process is repeated until all the synonyms are exhausted.

Next, FIG. 7 shows an embodiment in which the embodiment shown in FIG. 1 is used as a natural language processing device, and this device is used as a preprocessing device to configure a natural language question answering device. In FIG. 7, IO is a natural language processing unit having the same configuration as the embodiment shown in FIG. Also, // is the search/registration section,
/2 is a knowledge database, and /3 is a response sentence creation section.

The search/registration unit // receives the output from the natural language processing unit IO, that is, the semantic structure of the input sentence, and registers or searches the data in the knowledge database 7.2 according to the content of the sentence. stores the contents of input sentences as data. The response sentence creation unit/3 creates a response sentence according to the content of the input sentence.

The process will be explained below. For example, for the above-mentioned input sentence ``Atoms constitute molecules.'', the result of analyzing the sentence is the semantic structure (knowledge data).

Assume that 9 atoms (9 molecules)) are given to the search/registration section //. Here, the search/registration unit l/ determines whether the received sentence is an interrogative sentence or a declarative sentence, and if it is a declarative sentence, data with the same meaning or similar meaning as this new knowledge data input into the knowledge database 7.2 is determined. Search to see if it is already registered,
If it is not registered, register the intervened knowledge data in Knowledge Database/2 (if the already registered data has the same or similar meaning, there is no need to register it)
.

Here, checking whether data having the same meaning or similar meaning as the newly input knowledge data has already been registered is performed by checking the result of analysis of the input sentence (constructing).

A synonym table of knowledge data (consisting of two molecules, atoms) with synonyms and similar meanings! It is only necessary to generate all the outputs of the natural language processing unit 10 by referring to the above, and check whether there is data in the knowledge database that matches (matches) any of these synonymous/synonymous data.

Next, when the input sentence ``What does a molecule consist of?'' is given, it is similarly analyzed by the natural language processing unit IO, and the semantic structure (knowledge data) is given, for example, two molecules consisting of (, what (
Interrogative word)) is handed over to the search/registration department//. The search/registration section // recognizes that this knowledge data is an interrogative sentence, and searches for the word that corresponds to the interrogative "what" part.
Searching for data in the knowledge database 12 (in this case, it can be easily done by pattern matching processing using the knowledge data of the input sentence (two molecules consisting of X). Note that what is X (an interrogative word)? )I do. Here, the knowledge database/2 contains knowledge data (configure) corresponding to the above-mentioned input sentence "Atoms constitute molecules."

If two atoms (two molecules) are registered, there is no knowledge data to be matched as is, but as mentioned in the explanation of the embodiment shown in FIG. Then, by referring to the synonym table j, (consists of).

Molecule, what (interrogative word)) and (constituting. what (interrogative word) one molecule) are also generated at the same time, so this is matched with the knowledge data mentioned above, and the solution ``atom'' (what (interrogative word) = atom) is generated. is found. Based on this result (what = atoms), the response sentence creation section/
Step 3 replaces "what" in the input sentence with "atom" and then removes "ka" which indicates the meaning of the question to create the response sentence "Molecules are made up of atoms" and output it to a response device such as a display. do.

In addition, in the explanation of the example in Figure 1 and the example in Figure 7, in order to simplify the explanation of the semantic structure (knowledge data) as a result of analyzing the input sentence, the dependency relationship (predicate , subject, modifier), but the number of these elements is not limited to just three, and the number of elements may increase if the content is specified in more detail. Needless to say. This includes the present tense for the meaning interpretation of sentences.

There may be cases in which various conditions are included, such as tenses such as past tense, or conditions for the content of the sentence to be true.

In addition, "same meaning/similar meaning" in the present invention does not mean only words that have the same meaning or similar meaning as shown in the examples, but also includes, for example, "sell" and "buy" or "teach" and "teach". The same phenomenon as ``ra sells a book to b.''

” and [b buys a book from a. ", "A teaches English to B." and "A is taught English by B." etc.) also include the relationships of different expressions that arise depending on whether one focuses on the subject or the object.

〔Effect of the invention〕

As explained above, the present invention is equipped with a natural language processing unit that has data on synonyms and synonyms of words including important terms in sentence analysis, including the surface case relationships governing each term. Therefore, although the predicates have similar meanings (synonymous), the surface case structure governing the predicates is different, such as "A constitutes B." and "B consists of A." In this case, it is possible to treat the meanings of the sentences as similar (or the same). In addition, if a natural language processing device having such a function is applied to a question answering system, it will be able to handle a variety of expressions and realize more natural interaction for humans. It goes without saying that the functions of the present invention can be applied to information retrieval systems, document processing systems, and other fields where input/output using natural language is desired, and the effects are also the same.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing an embodiment of the natural language processing device of the present invention,
Figure 2 shows an example of the structure of a word dictionary; Figures 3 and 3 show examples of general syntax analysis results; Figure 3 shows an example of the contents of a synonym table; The figure is a flowchart showing the process of generating synonymous/similar expressions, and FIG. 7 is a diagram showing an embodiment of the question answering system to which the present invention is applied. /...input section, 2...sentence analysis section, 3...word dictionary, g...semantic structure extraction section,! ...synonym table,
10...Natural language processing unit, //...Search/registration unit,
/2...Knowledge database, /3...Response sentence creation section. Mine'3 Homo 4-V Hara j praise, ba) - Master's opposition (Ya Nuo) (K'ro) Conj (phantom -Itr-or Tro)

Claims (2)

[Claims] (1) An input unit that inputs sentences using natural language, a word dictionary that stores the relationship between heading words and part-of-speech information, and heading words, words that are synonymous with or similar to the heading words, and words thereof. A synonym table that stores case structure information dominated by A sentence analysis unit that performs synonymous/similar meaning analysis of the input sentence based on word information that has a synonymous/similar meaning relationship with the analysis result of the sentence analysis unit and case structure information that governs the relationship between each word. 1. A natural language processing device comprising: a semantic structure extraction unit that generates an expression and extracts semantic content. (2) An input unit that inputs sentences using natural language, a word dictionary that stores the relationship between words in the heading and part-of-speech information, and words in the heading and words that are synonymous with or similar to the word and those words. A synonym table that stores case structure information dominated by A sentence analysis unit that performs synonymous/similar meaning analysis of the input sentence based on word information that has a synonymous/similar meaning relationship with the analysis result of the sentence analysis unit and case structure information that governs the relationship between each word. There is a natural language processing unit equipped with a semantic structure extraction unit that generates an expression and extracts the semantic content, and the semantic structure (synonymous/similar expression) of the input sentence that is output from the natural language processing unit and semantic content) and information from a knowledge database that stores the semantic structure of sentences in advance. A natural language question answering device comprising: a search registration section that registers or searches for data; and a response sentence creation section that creates a response sentence in accordance with the content of the input sentence.