JP4114580B2 - Natural language processing system, natural language processing method, and computer program - Google Patents

Description

  The present invention relates to a natural language processing system, a natural language processing method, and a computer program for mathematically handling a natural language used by humans for daily communication, and in particular, to analyze syntax and semantics of a natural language sentence. The present invention relates to a natural language processing system, a natural language processing method, and a computer program.

  More specifically, the present invention relates to a natural language processing system, a natural language processing method, and a computer that output a result of syntactic / semantic analysis at a faster analysis speed with respect to a sentence including a compound word composed of a plurality of words. A natural language processing system and a natural language processing method for outputting a syntax / semantic analysis result with a higher recall when combining compound words into one to improve the analysis speed, and a computer, Regarding the program.

  Words that humans use for everyday communication, such as Japanese and English, are called “natural languages”. Many natural languages have a naturally occurring origin and have evolved with the history of mankind, people and society. Of course, people can communicate with each other by gestures and hand gestures, but natural language can realize the most natural and advanced communication.

  On the other hand, with the development of information technology, computers have become established in human society and have deeply penetrated into various industries and daily life. Now, not only computer data, but almost all information content such as images and sounds are handled on the computer, making it possible to perform advanced processing such as editing / processing, storage, management, transmission and sharing of information. .

  For example, a natural language written in various languages such as Japanese and English is inherently abstract and ambiguous, but can be processed computerically by handling sentences mathematically. . As a result, various applications / services related to natural language are realized by automated processing such as machine translation, dialogue system, search system, and question answering system.

  Such natural language processing is generally divided into processing phases of morphological analysis, syntax analysis, semantic analysis, and context analysis.

  In morpheme analysis, a sentence is segmented into morphemes which are the smallest semantic units, and part-of-speech recognition processing is performed. In syntax analysis, sentence structure such as phrase structure is analyzed based on grammatical rules. Since the grammatical rule is a tree structure, the parsing result generally has a tree structure in which individual morphemes are joined based on a dependency relationship. In semantic analysis, a semantic structure that expresses the meaning conveyed by a sentence is obtained based on the meaning (concept) of the words in the sentence and the semantic relationship between words, and the semantic structure is synthesized. In context analysis, a sentence series (discourse) is regarded as a basic unit of analysis, and a discourse structure is constructed by obtaining a semantic group between sentences.

  In particular, syntactic analysis and semantic analysis are indispensable techniques for realizing applications such as dialog systems, machine translation, document proofreading, and document summarization in the field of natural language processing.

  In the syntax analysis, a natural language sentence is received, and a dependency relationship between words (sentences) is determined based on grammatical rules. The parsing result can be expressed in the form of a tree structure (dependency tree) called a dependency structure. In the semantic analysis, it is possible to perform a process of determining a case relationship in a sentence based on a dependency relationship between words (sentences). The case relationship here refers to a grammatical role such as a subject (SUBJ) and an object (OBJ) possessed by each element constituting a sentence. In addition, semantic analysis may include processing for determining sentence tense, appearance, speech, and the like.

  By the way, the time required for syntax / semantic analysis in natural language processing is assumed to increase exponentially with respect to the number of morphemes contained in a sentence. For this reason, improvement in analysis speed can be expected by combining a plurality of morphemes into one and reducing the number of morphemes.

  For example, when a compound word composed of a plurality of words appears in syntax analysis or semantic analysis, a process of treating these words as one word, that is, a process of collecting a plurality of continuous morphemes Therefore, the analysis speed is improved.

  For example, a machine translation device has been proposed that processes a set of expression forms in a source text such as collocations as a single morpheme to improve the accuracy of syntactic analysis and reduce the time required for syntactic analysis. (For example, refer to Patent Document 1). In this case, an English-Japanese collocation dictionary is prepared in the HD device, and a group of expression forms represented by so-called collocations are stored in the English-Japanese collocation dictionary. Then, in the syntactic analysis process, an expression form composed of words combined by equivalence conjunctions is searched in an English text and registered in the English collocation dictionary, or the words constituting the searched expression form If the prefixes or suffixes are the same, the retrieved expression form is recognized as one morpheme and the syntax is analyzed without separation.

  In addition, a syntax analysis method that can efficiently analyze compound words, heavy sentences, and compound sentences that have been difficult to analyze in a natural language analysis apparatus using a computer has been proposed (see, for example, Patent Document 2). In this case, a combination of word strings whose part-of-speech information is verb + connecting particle + verb is defined as one verb. If a word having an attribute of “and” or “or” is detected with reference to a dictionary prepared in advance, the process proceeds as one word including words positioned before and after the word. Also, if at least one word selected from adjectives, adverbs, and exclamations is detected, the word modifies the first statement statement (a word indicating an event such as a verb or noun) that appears after the word. Proceed as a word to be processed.

  The compound words referred to in this specification include “compound nouns” composed of a plurality of nouns, “compound verbs” composed of a plurality of verbs, and the like.

  For example, as shown in the following example sentence (1), four consecutive nouns “youth”, “general”, “physical education”, and “meeting” are treated as one compound noun.

(1) A youth sports competition was held in Yokohama.
A youth sports competition was held in Yokohama.
→ A youth sports competition was held in Yokohama

  However, problems may occur by collecting different morphemes. FIG. 8 shows an example of the syntactic and semantic analysis result for the above example sentence. For example, even if an attempt is made to search for the result of this analysis with the keyword “meeting”, it does not match the above word (“Youth General Athletic Meet”) that treats consecutive nouns as one noun.

(2) What is the tournament held in Yokohama?

  For example, (1) cannot be adopted as an answer to an inquiry in natural language such as example sentence (2). FIG. 9 shows the syntactic and semantic analysis result for the example sentence (2). However, the analysis result shown in FIG. 8 is summarized as a compound noun and cannot be associated. Yes. In short, as a harmful effect of collecting continuous morphemes, the recall rate of the search system is reduced.

  Here, in order to maintain the reproduction rate, even if the search is performed based on a policy of taking a partial match instead of a complete match of the character string, another problem is caused.

(3) Where was the coalescence done?

  For example, in the inquiry sentence like the above example sentence (3), the word “union” partially matches with “youth general athletic meet”. FIG. 10 shows the syntactic and semantic analysis result for the example sentence (2), but the example sentence (1) is adopted as an answer because it is associated with the analysis result shown in FIG. In other words, the relevance rate of the search system is reduced.

JP 11-329178 A JP 2001-125898 A

  An object of the present invention is to provide an excellent natural language processing system and natural language processing capable of outputting a syntactic / semantic analysis result at a faster analysis speed with respect to a sentence including a compound word composed of a plurality of words. It is to provide a method and a computer program.

  A further object of the present invention is to provide an excellent natural language processing system and natural language processing capable of outputting a highly accurate syntax / semantic analysis result when combining compound words into one for improving the analysis speed. It is to provide a method and a computer program.

  A further object of the present invention is to output a high-accuracy syntax / semantic analysis result without reducing the recall rate and relevance rate of the search system when compound words are combined into one to improve the analysis speed. An object of the present invention is to provide an excellent natural language processing system, natural language processing method, and computer program.

The present invention has been made in consideration of the above problems, and a first aspect thereof is a natural language for analyzing a natural language sentence in which a compound word composed of a plurality of words composed of specific parts of speech appears. A processing system,
Means for acquiring a morpheme analysis result including a recognition result of a part of speech for each morpheme for the input natural language sentence;
Based on the morpheme analysis result, means for extracting a part where the morpheme of the specific part of speech is continuous in the input natural language sentence;
Means for providing a delimiter indicating that the extracted continuous morphemes of the specific part of speech are connected,
It is a natural language processing system characterized by comprising.

  Here, the natural language processing system according to the present invention outputs a continuous morpheme of the specific part of speech as a single word, that is, a compound word, given a delimiter indicating that it is connected between the morpheme. To do.

  And the specific part of speech said here is a noun and a verb, for example, and a compound word points out a compound noun and a compound verb.

  According to the present invention, when consecutive morphemes are formed together in the preprocessing of syntax / semantic analysis to form one compound word, a delimiter character indicating connection is provided between the original morphemes and the morphemes. Since the original morpheme can be easily extracted from the compound words grouped into one by the delimiter as described above, the recall in the search system can be maintained.

A second aspect of the present invention is a computer configured to execute, on a computer system, a process for analyzing a natural language sentence in which a compound word composed of a plurality of words composed of specific parts of speech appears. A computer program written in a readable format,
Obtaining a morphological analysis result including a recognition result of part of speech for each morpheme for the input natural language sentence;
Based on the morphological analysis result, extracting a portion where the morpheme of the specific part of speech is continuous in the input natural language sentence;
Providing a delimiter indicating concatenation between the extracted consecutive morphemes of the specific part of speech;
Outputting as a single word consecutive morphemes of the specific part-of-speech given a delimiter indicating that they are connected between the morphemes;
A computer program characterized by comprising:

  The computer program according to the second aspect of the present invention defines a computer program described in a computer-readable format so as to realize predetermined processing on a computer system. In other words, by installing the computer program according to the second aspect of the present invention in the computer system, a cooperative action is exhibited on the computer system, and the natural language according to the first aspect of the present invention. The same effects as the processing system can be obtained.

  Advantageous Effects of Invention According to the present invention, an excellent natural language processing system and natural language processing capable of outputting a syntax / semantic analysis result at a faster analysis speed for a sentence including a compound word composed of a plurality of words connected in series. Methods and computer programs can be provided.

  In addition, according to the present invention, when compound words are combined into one for improving the analysis speed, a highly accurate syntax / semantic analysis result can be obtained without reducing the recall rate and the matching rate of the search system. An excellent natural language processing system, natural language processing method, and computer program that can be output can be provided.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The natural language processing system according to the present invention performs a process of collecting compound words composed of continuous morphemes such as compound nouns and compound verbs into one in order to improve the analysis speed. It is possible to output high-accuracy syntax / semantic analysis results without reducing the rate and precision.

  Here, Lexical Functional Grammar (LFG) can be cited as a representative example of grammar theory for performing syntax / semantic analysis. The present invention can be implemented by being incorporated into syntactic / semantic analysis processing based on, for example, LFG grammar theory. In LFG, linguistic knowledge, that is, grammar of native speakers is configured as a component separated from computer processing and other non-grammatical processing parameters that affect the processing operation of the computer.

  First, an overview of the natural language processing system will be briefly described. FIG. 1 schematically shows a configuration of a natural language processing system 1 based on LFG. This natural language processing system can be realized in the form of executing a predetermined natural language processing application program on a general computer system such as a personal computer (PC).

  The morpheme analysis unit 2 has a morpheme rule 2A and a morpheme dictionary 2B related to a specific language such as Japanese, and performs a part-of-speech recognition process by segmenting an input sentence into morphemes that are semantic minimum units. For example, if a sentence “My daughter speaks English” is input, “{up} daughter {Noun} of I {Noun} {up} English {Noun} {up} } Speak {Verb1} {tr} mass {jp}. {Pt} "is output.

  Such a morphological analysis result is then input to the syntactic / semantic analysis unit 3. The syntactic / semantic analysis unit 3 has dictionaries such as a grammar rule 3A and a valence dictionary 3B, and is based on the analysis of phrase structure based on the grammar rule, the meaning of words in a sentence, and the semantic relationship between words. Analyzing the semantic structure expressing the meaning conveyed by the sentence (The valence dictionary describes the relationship between verbs and other components in the sentence such as the subject, and the semantic relation between the predicate and the related word. Can be extracted). As a result of parsing, “c-structure (constituent structure)” representing a phrase structure of a sentence including words and morphemes as a tree structure, and an input sentence based on a case structure such as a subject and an object are questioned. “F-structure (functional structure)” is output as a result of semantic and functional analysis such as sentences, past tense, and polite sentences.

  FIGS. 2 and 3 respectively show c-structure and f-structure obtained as a result of processing the input sentence “My daughter speaks English” by the syntactic / semantic analysis unit 1.

  c-structure represents the structure of words and phrases in a sentence in a tree structure format, and is defined by a syntax category. For example, phonological interpretation for generating a phoneme string can be performed based on c-structure. On the other hand, f-structure clearly expresses a grammatical function, and includes a grammatical function name, a semantic form, and a feature symbol. By referring to f-structure, it is possible to obtain an understanding of the meaning of a subject, an object, an complement, a modifier, and so on. The f-structure is a set of features attached to each node of the c-structure, and is expressed in the form of an attribute-attribute value matrix as shown in FIG. That is, the left side in [] is a feature (attribute) name, and the right side is a feature value (attribute value).

  For details of LFG, see, for example, R.A. M.M. Kaplan and J.H. Bresnan co-author of the paper. "Lexical-Functional Grammar: A Formal System for Grammatical Representation" (The MIT Press, Cambridge (1982) Reprinted in Formal Issues in Lexical-Functional Grammar, pp.29-130.CSLI publications, Stanford University (1995 ).) Etc.

  Next, a detailed description will be given of the grouping process for compound words composed of continuous morphemes such as compound nouns by natural language processing according to the present invention.

  As already mentioned in the section of “Background Art”, by combining the compound words as one word, the analysis speed at the time of syntax / semantic analysis is improved, but the recall or relevance rate in the search system is reduced. Accompanied by evil. Therefore, in the present invention, when forming a single compound word by concatenating consecutive morphemes in the preprocessing of syntax / semantic analysis, a delimiter indicating that the original morphemes are connected is given. did. As described above, since the original morpheme can be easily extracted from the morpheme analysis result by the delimiter character, the reproducibility in the search system can be maintained.

  FIG. 4 shows, in the form of a flow chart, a processing procedure in which a plurality of consecutive morphemes are combined into one compound word based on the result of morphological analysis as preprocessing for syntax / semantic analysis. However, compound nouns are handled here as examples of compound words. Also, when forming a compound word, some delimiter is inserted between morphemes.

  First, the original Japanese original is input, and a morphological analysis result obtained from a morphological analysis process performed separately is acquired (step S1). In the morphological analysis, the part of speech is segmented into morphemes, which are the smallest semantic units, and the part of speech is recognized.

  Next, while substituting 1 for the variable i and until i reaches the number of morphemes contained in the input sentence, in loop 1, when combining multiple morphemes into one compound word, Processing to insert some delimiter is performed. In loop 1, a buffer is prepared for sequentially writing continuous morphemes constituting a compound word.

  When the i-th morpheme is a noun (step S2), in loop 2, when a compound word including the morpheme is formed, a process of inserting a delimiter between the morpheme and the morpheme is performed. That is, when the buffer is not empty and one or more morphemes have already been accumulated (step S3), a delimiter (in this embodiment, “/” (slash) is used as a delimiter between the previous morphemes. Is inserted (step S9), and then written into the buffer to connect the morphemes (step S4).

  Next, the variable i is incremented by 1 (step S5), and the next morpheme is extracted from the input sentence.

  If the i-th morpheme is not a noun, the loop 2 is exited to determine whether the buffer is empty, that is, whether a compound noun is formed (step S5). If a compound noun is stored in the buffer, it is taken out as one morpheme and then returned to an empty state (step S10). The extracted compound noun is in a state in which a delimiter is inserted between the original morpheme.

  Then, the i-th morpheme subjected to the above processing is output (step S7), i is incremented by 1, and loop 1 is repeatedly executed until the processing is completed for all the morphemes.

  Next, the pre-processing of syntax / semantic analysis according to the present embodiment will be specifically described based on the example sentence (1).

  When an input sentence is input to morphological analysis, a character string is acquired together with the part-of-speech information as an output result, and is held as a morphological analysis result. Here, the morphemes “Yokohama”, “De”, “Youth”, “Comprehensive”, “Physical education”, “Meeting” “Ga”, “Done”, “Red”, “Ta” that compose the input sentence. 5, part-of-speech information and the order from the beginning of the sentence are respectively stored. In the figure, “surface layer” is a character string delimited by morpheme from the original text, and “entry word” is the original form when the morpheme is a usage word.

  Next, the part-of-speech information of the i-th morpheme is referenced, and if it is not a noun, it is output together with a blank character. When i = 1, since the i-th morpheme “Yokohama” is a noun (step S2) and the buffer is empty (step S3), “Yokohama” is stored in the buffer as it is (step S4). Then, the variable i is set to i + 1 (step S5).

  Since the next morpheme i is “de” and the part of speech information is “case particle” (step S2), the buffer character string is output together with a blank character, and the buffer is emptied (step S10). Subsequently, the i-th morpheme (“de”) is output (step S7). Then, the variable i is set to i + 1 (step S8).

  Since the next morpheme i is “youth”, the part of speech is “noun” (step S2), and the buffer is empty (step S3), “youth” is stored in the buffer as it is (step S4). Then, i is set to i + 1 (step S5).

  The next morpheme i is “general” and the part of speech is “noun” (step S2). At this time, since the buffer is not empty (step S3), a delimiter (here, “/”) and the i-th morpheme “total” are added to the buffer (step S9). Therefore, what is stored in the buffer here is “Youth / Comprehensive”. Then, the variable i is set to i + 1 (step S5). Since the next morpheme i is “physical education”, the part of speech is “noun” (step S2), and the buffer is not empty (step S3), the procedure in loop 2 is repeated. That is, the character string in the buffer is “Youth / Comprehensive / Physical Education” (step S9).

  As long as the part-of-speech information of the subsequent i-th morpheme is a noun, the processing in the loop 2 is repeated. Here, since the next morpheme i is “meeting” and the part of speech is “noun” (step S3), the same processing is performed. Therefore, the contents of the buffer, that is, the composition of the compound noun is “Youth / Comprehensive / Physical Education / Meeting” as shown in FIG.

  Further, since the next morpheme i is “ga” and the part of speech is “case particle” (step S2), the repetition in the loop 2 is stopped, the buffer character string is output together with a blank character, and then the morpheme i is output. (Step S7).

  Further, the variable i is set to i + 1 (step S8), and the loop 1 is repeated. Thereafter, morphemes other than nouns “do”, “re”, and “ta” are output (step S7).

  As described above, according to the pre-processing procedure for syntax / semantic analysis according to the present embodiment, “youth / general / physical education / competition is performed in Yokohama” is output in response to the input of example sentence (1). Is done.

  FIG. 7 shows an example of the syntactic and semantic analysis result output by the processing procedure shown in FIG. 4 with the character string consisting of the example sentence (1) as an input.

  As shown in the figure, if the search system is applied using the analysis results, it is possible to achieve an appropriate match with a search term such as “meeting” and at the same time inappropriate with a search term such as “union” It becomes possible to prevent a coincidence. In other words, it is possible to output a highly accurate syntax / semantic analysis result without lowering the recall rate or relevance rate of the search system.

[Supplement]
The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention.

  Although the present embodiment has been described based on the LFG grammar theory, of course, the present invention can be similarly applied to an analysis system having other grammar rules.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims section described at the beginning should be considered.

FIG. 1 is a diagram schematically showing a configuration of a natural language processing system 1 based on LFG. FIG. 2 is a diagram showing c-structure obtained as a result of processing the input sentence “My daughter speaks English” by the syntactic / semantic analysis unit 1. FIG. 3 is a diagram showing f-structure obtained as a result of processing the input sentence “My daughter speaks English” by the syntactic / semantic analysis unit 1. FIG. 4 is a flowchart showing a procedure of processing for collecting a plurality of continuous morphemes into one compound word based on the result of morphological analysis as preprocessing for syntax / semantic analysis. FIG. 5 is a diagram showing a morphological analysis result for the example sentence (1). FIG. 6 is a diagram showing the composition of compound nouns obtained from the example sentence (1) by preprocessing of syntax / semantic analysis. FIG. 7 is a diagram showing an example of the result of syntactic and semantic analysis that is output by the processing procedure shown in FIG. 4 with the character string consisting of the example sentence (1) as an input. FIG. 8 is a diagram illustrating an example of a syntax and semantic analysis result for the example sentence (1). FIG. 9 is a diagram showing the result of syntactic and semantic analysis for the example sentence (2). FIG. 10 is a diagram illustrating a syntax-meaning analysis result for the example sentence (3).

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Natural language processing system 2 ... Morphological analysis part 2A ... Morphological rule, 2B ... Morphological dictionary 3 ... Syntactic / semantic analysis part 3A ... Grammar rule, 3B ... Joint value dictionary

Claims (4)

A natural language processing system for analyzing a natural language sentence in which a compound word composed of a plurality of words composed of specific parts of speech appears,
Means for acquiring a morpheme analysis result including a recognition result of a part of speech for each morpheme for the input natural language sentence;
Based on the morpheme analysis result, means for extracting a part where the morpheme of the specific part of speech is continuous in the input natural language sentence;
Means for providing a delimiter indicating that the extracted continuous morphemes of the specific part of speech are connected,
Means for outputting as a single word consecutive morphemes of the specific part of speech, given a delimiter indicating that the morpheme is connected between the morphemes;
Means for searching for a match with a search word for each original morpheme delimited by the delimiter for a compound word given a delimiter indicating that the morpheme is connected between the morphemes;
A natural language processing system comprising: The specific part of speech is a noun or a verb,
The natural language processing system according to claim 1. A natural language processing method for analyzing a natural language sentence in which a compound word composed of a plurality of words composed of specific parts of speech appears on a natural language processing system constructed using a computer,
The morpheme analysis result acquisition means provided in the computer acquires a morpheme analysis result including a recognition result of a part of speech for each morpheme for the input natural language sentence;
An extraction means provided in the computer, based on the morpheme analysis result, extracting a portion where the morpheme of the specific part of speech is continuous in the input natural language sentence;
A step of providing a delimiter indicating that the assigning means included in the computer is connected between the extracted morphemes of the specific parts of speech that are consecutive;
The output means provided in the computer outputs the continuous morphemes of the specific part of speech as a single word given a delimiter indicating that the morpheme is connected between the morphemes;
The search means provided in the computer matches a search word for each original morpheme delimited by the delimiter for a compound word given a delimiter indicating that it is connected between the morphemes The step of searching to take
A natural language processing method comprising: The specific part of speech is a noun or a verb,
The natural language processing method according to claim 3.

Images