JP2011145844A - Predicate functional expression normalization method, device and program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To convert a predicate as the center of sentence information in natural language processing into the simplest form without changing the meaning as much as possible. <P>SOLUTION: With respect to the word information of each morpheme corresponding to the predicate of an input sentence analyzed by a morphemic analysis part 2, and extracted by a predicate extraction part 3, a meaning label application part 4 applies a meaning label to the word information of a function word string configuring functional expression by using a functional expression meaning label dictionary 1 configured by combining at least a meaning label expressing the meaning of functional expression giving an influence to the meaning of the predicate and the list of the functional expression corresponding to each meaning label, and an NULL deletion part 5 and a redundant rule application part 6 delete a function word and a redundant function word which do not give any influence to the meaning of the predicate according to the pertinent meaning label, and a practical use generation part 7 generates the predicate of the input sentence based on the word information of each morpheme corresponding to the residual predicate of the input sentence. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for normalizing a predicate that is the center of sentence information in natural language processing, that is, converting it to the simplest form without changing the meaning of the predicate itself as much as possible.

<Description of task>
In order to obtain useful information from an enormous amount of documents such as minutes, questionnaires, and web text, it is essential to automatically extract and aggregate information such as “who and what” from the target document. This technique is called text mining. In this text mining, unlike the extraction of specific names such as “who”, “where”, and “what”, it is difficult to extract and aggregate “predicates” corresponding to “what”. The reason is that the predicate is not simply expressed by a single word.

For example, if there is a sentence that says “My computer is broken,” the “broken” that hits “What” is the verb “broken”, the auxiliary verbs “chu” “ta”, and the final particle “yo”. It is composed of a combination. In this way, the predicate representing the meaning of “what” is composed of a combination of “content words” such as verbs, nouns, adjectives, adjective verbs and adverbs and “function words” such as particles and auxiliary verbs. When the sentence “PC is broken” is divided into “content word” and “function word”,
It's broken = broken (ru) [content word] + cha [function word] + ta [function word] + yo [function word] (1)
It becomes.

  One of the major reasons for the difference in the surface form (notation) of predicates is the presence of “function words” in the predicates. “Cha” of the predicate “broken” or “has broken” of the predicate “broken” is a sequence of a plurality of function words (hereinafter, one or more function words in the present invention). The character string consisting of is called “functional expression”), and varies greatly depending on the style and style (honorifics and colloquial language). Therefore, the same predicate meaning "broken" is expressed in various ways such as "broken", "broken", "broken", and the mining failure of "same meaning" become.

However, these “functional expressions” are important clues to distinguish different meanings. For example, “broken” represents the same thing as “has broken” but different from “not broken”. Even if the same verb “break” is used, the case where the functional expression “Cha” is added and the case where “Tenai” is added are completely opposite. If this is expressed by "content word" and "functional expression",
Broken (Ru) [content word] + Chaotic [Functional expression] ≠ Broken (Ru) [Content word] + Unbreakable [Functional expression] (2)
It becomes.

  In this way, functional expression is an obstacle to grouping predicates with the same meaning, but on the other hand, it is an important clue when differentiating predicates with different meanings. In other words, in order to improve the accuracy of text mining, such as extracting and counting predicates, “the functional expressions with the same meaning but different surface forms” are summarized and “functional expressions necessary to distinguish the meanings of predicates” "Requires predicate normalization to remain.

<Prior Art 1>
Non-patent document 1 discloses a mining technique for extracting and counting predicates corresponding to “what” from text. Non-Patent Document 1 proposes a technique for automatically extracting and tabulating three pieces of information “situation”, “action”, and “subjectivity” from a blog. And the extraction of predicates corresponding to “what” is incorporated as part of “behavior” extraction.

  However, in the technique of Non-Patent Document 1, since the sentence is morphologically analyzed, and only the normal form of the verb is extracted as the normal form of the predicate, “broken”, “not broken”, “looks broken”, etc. All predicates that represent different meanings depending on the functional expression are also extracted as “broken”. Therefore, the accuracy of text mining is reduced. FIG. 1 shows an example of a predicate extraction result by the method of Non-Patent Document 1. “/” Indicates a morpheme (word) unit break of the function word.

<Conventional technology 2>
As another method of predicate normalization including function expression, one disclosed in Non-Patent Document 2 can be cited. Non-Patent Document 2 performs paraphrasing of function expressions by associating phrase function expressions with one-to-one or one-to-N function expressions. For example, when a predicate such as “broken / chat / ta” is input, a one-to-one paraphrase such as “chat” → “tesama” is performed, such as “broken / tame / ta”. By outputting the paraphrase candidates for function expression in this way, it is possible to combine all the output as the paraphrase destination as “same meaning”.

  However, in Non-Patent Document 2, the function words can be accurately rephrased in a one-to-one or one-to-N correspondence, so the granularity of predicate normalization that “summarizes” predicates representing the same event necessary for mining is as follows: too detailed. Non-Patent Document 2 also performs paraphrasing in a one-to-N correspondence, but it adds elements to maintain the grammatical connection relationship after paraphrasing, and simplifies “What is the key?” It is not a paraphrase to do. FIG. 2 shows an example of an output result in the case where the method of Non-Patent Document 2 is used in other words.

  In the case of FIG. 2, the input predicates all represent “an event that something is broken” that is “broken”, but with the technique of Non-Patent Document 2, even the function words that are unnecessary for text mining are accurate. In other words, predicates do not come together. This is because the final particle “Yo” and “Teshima [completed]” and “Ta [completed]” in which past tenses are duplicated are all rephrased. The normalization required for text mining is the simplest normalization that leaves only the functional expression related to “whether or not it represents the same event”. Since it is too fine, it causes a reduction in the reproducibility of mining (that is, the rate of collecting the same meaning).

Ken Kurashima, Kou Fujimura, Hidenori Okuda “Experience Mining from Large Scale Texts” Proceedings of the 19th Data Engineering Workshop of the IEICE, DEWS2008 A1-4, 2008, pp.301-310 Satoshi Matsuyoshi and Satoshi Sato “Rephrasing Japanese Functional Expressions that Control Style and Difficulty” Natural Language Processing, Vol.15, No.2, 2008, pp.75-99

  As described above, the conventional technique has the following problems in predicate extraction and aggregation.

  1. If only the verbs ignoring the functional expression are extracted, the predicates having different meanings are extracted as “same”, so that the accuracy of text mining is reduced (prior art 1).

  2. In other words, the predicate that takes into account all functional expressions reduces the reproducibility because even predicates that represent “the same event” retain functional expressions that are unnecessary for text mining (Prior Art 2). ).

  In order to solve two problems in predicate normalization for predicate extraction and aggregation, the present invention takes the following method.

  a. The meaning of the function expression that directly affects the meaning of the predicate is expressed using three indices, and a function expression meaning label (hereinafter simply referred to as “meaning label”) is created based on the indices.

  b. A list of function word strings is associated with the meaning label to form a function expression meaning label dictionary, and among the function words constituting the function expression in the predicate, a meaning label is assigned to the function word that matches the entry in the dictionary.

  c. Delete function words that have not been given a semantic label.

  d. When the function word to which the semantic label is assigned has redundancy, the function word is deleted according to a predetermined redundancy rule.

  By taking such a method, it is possible to normalize the functional expression into the simplest form while retaining the functional expression that affects the meaning of the predicate. As a result, it is possible to improve both the accuracy and recall of predicate extraction and tabulation in text mining and the like.

According to the present invention,
For the functional expression of the predicate, based on the semantic label, function words that do not affect the meaning of the predicate and redundant function words are deleted to simplify the predicate. As a result, it is possible to normalize the functional expression that causes the different predicate surface layer forms to the simplest form, and to improve the reproducibility of text mining for performing predicate aggregation and the like.

  Also, because semantic labels are attached to functional expressions based on the three indicators, predicates that change the meaning of events represented by functional expressions, such as “broken” and “not broken,” can be distinguished. it can. As a result, it is possible to improve the accuracy of tabulating the same meaning such as mining.

  In order to select the simplest functional expression and generate predicates, it can be widely used in mining and other natural language applications such as text summarization techniques.

Explanatory drawing which shows an example of the predicate extraction result by the prior art 1. Explanatory drawing which shows an example of the paraphrase result by the prior art 2. The block diagram which shows an example of embodiment of the predicate function expression normalization apparatus of this invention Explanatory drawing which shows an example of a function expression meaning label dictionary Flowchart of processing in the predicate function expression normalization apparatus of FIG. Explanatory drawing showing an example of morphological analysis results Explanatory diagram showing an example of predicate extraction results Explanatory drawing which shows an example of a meaning label assignment result Explanatory drawing which shows an example of a NULL deletion result Explanatory drawing which shows an example of a redundant label deletion result Explanatory drawing showing another example of morphological analysis results Explanatory drawing showing another example of predicate extraction results Explanatory drawing which shows the other example of a semantic label provision result Explanatory drawing which shows the other example of a NULL deletion result Explanatory drawing which shows the other example of a redundant label deletion result

  An object of the present invention is to normalize predicates that represent the same event but with different surface forms to the same surface form. For this reason, normalization is performed by leaving only functional expressions that directly affect the meaning of the event represented by the predicate. As a method of determining which function expression affects the meaning of the predicate, the present invention refers to a function expression meaning label dictionary created in advance according to the index described below, and A method is used in which semantic labels are assigned only to functional expressions that affect meaning, and the entire predicate is normalized using the semantic labels as clues. Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

  FIG. 3 shows an example of an embodiment of the predicate function expression normalization apparatus of the present invention. In the figure, 1 is a function expression meaning label dictionary, 2 is a morpheme analyzer, 3 is a predicate extractor, and 4 is A semantic label assignment unit, 5 is a NULL deletion unit, 6 is a redundancy rule application unit, and 7 is a utilization generation unit.

  Normally, a document containing a plurality of sentences is input to this device. However, since processing in each part is performed for each sentence, it is described in the following description that one sentence is input. To do.

  The functional expression meaning label dictionary 1 corresponds to a semantic label representing the meaning of the functional expression that affects the meaning of the predicate, a list of character strings in the standard form of the functional expression corresponding to each semantic label, and one predicate. When the function expression to be included includes a plurality of function words having the same semantic label, it is combined with a redundancy rule for each semantic label representing how to retain the function words, and an example is shown in FIG. .

  The function expression meaning label dictionary 1 is created as follows.

First, a semantic label is created along the following indices (A) to (C), and a list of character strings in a standard form of functional expression corresponding to each semantic label is given. By creating a semantic label along this index, it is possible to assign a semantic label only to “a functional expression that directly affects the meaning of the predicate”.
(A) Difference in tense: A semantic label characterizing the tense of the event represented by the predicate is included, and a difference such as “past” or “non-past” can be distinguished.
(B) Negative difference: A semantic label for distinguishing negative or positive of the event represented by the predicate is included, and a difference such as “negative” and “positive” can be distinguished.
(C) Modality difference: Contains a semantic label for distinguishing whether the event represented by the predicate includes the speaker's subjectivity (ie, modality expression), such as “subjectivity”, “objective”, etc. Can be distinguished.

  In this embodiment, 12 types of semantic labels are assigned as semantic labels that satisfy (A) to (C) to the standard form of functional expression. That is, as the twelve kinds of semantic labels, the meaning label “completed” is satisfied as satisfying (A) (that is, those given this label are “past”, those not given are “non-past”), and (B) The meaning label of “Negation” as satisfying (that is, “Negation” is given to this label and “Affirmation” is not given), “Question”, “Invitation / will”, “Wish” as satisfying (C) “Request”, “Recommend”, “Necessary”, “Permission”, “Inference”, “Possible” meaning labels (ie, those given this label are “subjective” and those not given are “objective”) as entries ( In addition, “solicitation / will” is counted as two types.)

  The number of types of semantic labels depends on the application. For example, when it is desired to distinguish the functional expressions “maybe”, “maybe”, “yoda” etc. corresponding to the semantic label “estimation” according to their “confidence”, “estimation 1” “estimation 2” Different semantic labels such as “estimation 3” may be created and assigned.

  Second, when the function expression corresponding to one predicate includes a plurality of function words having the same semantic label, a redundancy rule representing how to hold the function words is described for each semantic label. This redundancy rule is based on the knowledge of the present invention that "when there are a plurality of function words representing the same meaning in the same predicate, basically, the meaning represented by the predicate can be retained by leaving only one of them". Is.

  In the present embodiment, “the one representing the position of the semantic label to be left when there are a plurality of function words having the same semantic label in the same predicate” is used as the redundancy rule. That is, the redundancy rule “First” indicates that the function word that appears first in the predicate among the function words with the same meaning label is left, and the redundancy rule “Last” indicates the predicate among the function words with the same meaning label. This means that the function word that appears at the end of is left. However, this rule is “If (even), Delete All If (odd), First” when the semantic label is “Negative”. The function word is left, and if it is an even number, it will be deleted) ”. Also, if an exception occurs in the “First” and “Last” redundancy rules, an exception rule is assigned. In FIG. 4, in addition to “Last”, an exception rule “If (same surfs), First (leave the first function word when the surface shape is the same)” is added to the redundancy rule of the semantic label “complete”.

  Note that the indicators “difference in tense”, “difference in negative”, and “difference in modality” in FIG. 4 are described for convenience of explanation and are not necessarily included in the actual function expression meaning label dictionary 1. Absent. Further, the redundancy rule may be configured as a single table without being included in the function expression meaning label dictionary 1.

  The morpheme analysis unit 2 performs a well-known morpheme analysis process on an input sentence that is directly input from a keyboard or the like (not shown), read from a storage medium, or input from another device or the like via a communication medium. Word information such as notation, reading, standard form, part of speech, inflection type, inflection form, etc. for each morpheme is output to the predicate extraction unit 3.

  The predicate extraction unit 3 uses the word information for each morpheme corresponding to the content word and the functional expression that follows from the word information for each morpheme of the input sentence output from the morpheme analysis unit 2 as a predicate for the input sentence. Extracted and output to the semantic label assigning unit 4.

  The semantic label assigning unit 4 uses the functional expression semantic label dictionary 1 to indicate a functional word string that constitutes a functional expression among word information for each morpheme corresponding to the predicate of the input sentence output from the predicate extraction unit 3. A semantic label is attached (added) to the word information of the word information and output to the NULL deletion unit 5.

  The NULL deletion unit 5 deletes the word information of the function word to which the semantic label is not assigned from the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is output that is output from the semantic label addition unit 4. And output to the redundancy rule application unit 6.

  The redundancy rule application unit 6 uses a predetermined redundancy rule, here, the redundancy rule described in the functional expression meaning label dictionary 1, to the predicate of the input sentence after the meaning label is output from the NULL deletion unit 5. Of the word information for each corresponding morpheme, the word information of the function word excluded by the redundancy rule is deleted and output to the utilization generation unit 7.

  Here, the redundancy rule application unit 6 and the utilization generation unit 7 are functional words and redundancy that do not affect the meaning of the predicate among the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is given. A function word deletion unit is configured to delete word information of a simple function word.

  The utilization generation unit 7 outputs word information for each morpheme corresponding to the predicate of the input sentence after deleting the function word that does not affect the meaning of the previous description unit output from the redundancy rule application unit 6 and the redundant function word. Generate and output a predicate for the input statement based on

  FIG. 5 shows the flow of processing of the predicate function expression normalization apparatus described above, and the details of the configuration and operation of each part will be described below with specific examples.

<Input>
The input of the functional expression normalizer is a sentence written in Japanese. In the present embodiment, “the personal computer has been broken” will be described as an example of an input sentence.

<Morphological analysis>
In the morphological analysis unit 2, a known morphological analysis process is performed on the input sentence (s1). In morphological analysis, a sentence is divided into morpheme (word) units, and word information such as notation, reading, standard form, part of speech, inflection type, and inflection form is given to each word. A known morphological analyzer may be used. FIG. 6 shows an example of the morphological analysis result in this embodiment.

<Predicate extraction>
In the predicate extraction unit 3, a morpheme analysis result, that is, word information for each morpheme of the input sentence is input, and a predicate corresponding to “what” is extracted (s2). "Predicates" are "content words" whose part of speech is a verb, adjective, adjective verb or adverb, or a noun followed by an auxiliary verb such as "da", and the part of speech is a particle or auxiliary verb, or "chau" A function word that is a non-independent verb, noun, adjective, adjective verb or adverb (hereinafter referred to as a non-independent category) or a formal noun such as “koto” that cannot function as a content word by itself It consists of a combination with a “functional expression” consisting of two or more character strings.

  The predicate extraction unit 3 extracts word information for each morpheme corresponding to the content word and the functional expression that follows from the word information for each morpheme of the input sentence as a predicate of the input sentence. First, the word information for each morpheme of the input sentence and an arbitrary dictionary in which the standard form of the functional expression is described, here, the above-described functional expression meaning label dictionary 1 are collated with each other, and the matched ones are matched. If there is, the word information of the morpheme string is extracted as a functional expression, and then the word information of the morpheme whose part of speech is a particle or auxiliary verb, a category of independence, or a formal noun among the remaining word information for each morpheme. It is extracted as a functional expression, and finally, morpheme word information that is a noun followed by a verb, an adjective, an adjective verb or an adverb, or an auxiliary verb is extracted as a content word.

  In the case of the above-mentioned input sentence “PC has been broken”, “cha” and “ta” that match the list (entry) of character strings in the standard form of the functional expression in the functional expression meaning label dictionary 1 first. Next, “yo” belonging to the part of speech “particle” is recognized as a functional expression, and finally “broken” belonging to the part of speech “verb-independence” is recognized as a content word. As a result, “It's broken” is extracted as a predicate. FIG. 7 shows an example of the predicate extraction result in this embodiment.

<Semantic labeling>
In the semantic label assigning unit 4, word information for each morpheme corresponding to the predicate of the input sentence is input, and a functional expression that affects the meaning of the “event” represented by the predicate using the functional expression semantic label dictionary 1. A semantic label is assigned (s3). That is, the semantic label assigning unit 4 uses the functional expression semantic label dictionary 1 to assign a semantic label to the word information of the functional word string constituting the functional expression among the word information for each morpheme corresponding to the predicate of the input sentence. More specifically, the list of character strings in the standard form of the functional expression in the functional expression meaning label dictionary 1 is collated with the standard form in the word information of the functional word by the “longest matching method from the back”, Assign the corresponding semantic label.

  This semantic labeling process may be performed in the same manner as normal morphological analysis. For example, a method for determining whether or not two consecutive function words can be connected and assigning a semantic label string that can be connected, or a method for assigning the most likely semantic label using a probability model that represents the likelihood of a semantic label string Etc.

  At this time, in the semantic label assigning unit 4, the semantic label is “empty” for a functional word that does not have an entry corresponding to the functional expression semantic label dictionary 1, that is, a functional expression that does not affect the meaning of the predicate. A label such as “NULL” is attached.

  As described above, when the predicate is “broken”, according to the longest match method from the back, “yo” is first analyzed, but there is no entry in the functional expression meaning label dictionary 1, An empty semantic label "NULL" is given. Next, “ta” and “chau” are analyzed, and a “complete” semantic label is assigned to each. This semantic label assignment operation is terminated immediately before the last morpheme when the morpheme constituting the predicate is processed by the longest match method from the back, that is, immediately before the content word “break”. FIG. 8 shows an example of the meaning label assignment result in this embodiment.

<Function word deletion based on semantic labels>
<Deleting NULL>
Based on the semantic label attached to the predicate, normalization to the simplest predicate is performed. In the present invention, a method of “giving a semantic label only to a thing related to the meaning represented by the predicate” is taken along the indicators (A) to (C). Therefore, first, in the NULL deletion unit 5, word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is given as an input, and a function word to which no semantic label is assigned, here an empty semantic label “NULL” As for the function word to which "" is given, all the entries (word information) are deleted assuming that the meaning of the predicate is not affected (s4).

  As mentioned above, if the predicate's functional expression is “Cha-ta-yo”, “yo” with the empty semantic label “NULL” is deleted, and the function expression is simplified to “Cha-ta”. . FIG. 9 shows an example of a NULL deletion result in this embodiment.

<Delete redundant label>
Next, in order to convert the predicate into the simplest predicate, the redundant rule application unit 6 takes as input the word information for each morpheme corresponding to the predicate of the input sentence after giving the semantic label and after deleting NULL, and the function expression meaning Using the redundancy rule described in the label dictionary 1, all entries (word information) of redundant function words are deleted (s5).

  As described above, the redundancy rule is that when there are a plurality of function words representing the same meaning in the same predicate, the meaning represented by the predicate can be retained by leaving only one of them. It is based on. However, this rule changes to a rule of “remain in case of odd number and delete in case of even number” when the semantic label is “negative”.

  If the predicate after the semantic label is added and after NULL deletion is “broken”, the same meaning is included in the word information for each morpheme corresponding to the predicate after the semantic label is applied and after NULL is deleted. It is checked whether or not there are two or more labels. If two or more labels exist, deletion is performed according to the redundancy rule.

  Since the word information for each morpheme corresponding to the predicate “broken” in this embodiment has two semantic labels “complete”, it is subject to a redundancy rule. The redundancy rule of “complete” is “the first (if the surface layer shape is the same), the first (Last) is left otherwise”. Along with this, the function word “Cha” is deleted, and the function word “ta” with the simplest semantic label “complete” is left. FIG. 10 shows an example of the redundant label deletion result in this embodiment.

<Utilization generation>
Finally, in the utilization generation unit 7, word information for each morpheme corresponding to the predicate of the input statement after deleting the function word that does not affect the meaning of the predicate and the redundant function word is input, Connection is made to generate a predicate of the input statement (s6).

  In Japanese, it is necessary to use morphemes (words) in order to connect morphemes, but which one is used depends on the surface form and part of speech of the following word.

  For the morpheme connection process including the use of the word, a utilization generator based on a language model can be used. This is a generator that uses a model that has learned from the correct data in advance which surface type, part-of-speech and inflection type of the front word, and the surface form and part-of-speech of the rear word as features. It is. Based on this model, an optimal notation is generated when the surface form, part of speech, and utilization type of new and previous words are input.

  In the case of the last morpheme, there is no following word, so a morpheme (for example, a punctuation mark) indicating the end of a sentence is added at the time of connection generation, or “last function word is changed to a standard form” Add additional rules such as In addition to using a utilization generator based on a language model, predicates can also be generated based on utilization conversion rules.

  In the present embodiment, it is necessary to connect the content word “broken” and the function expression “ta”, but the “notation; part of speech” of the content word “broken” is added to the utilization generator based on the language model described above. ; "Breaking; verb-independence; one step" which is "utilization type", "ta; auxiliary verb; special-ta" of functional expression "ta", and ".; Symbol-punctuation" of morpheme "." ”Can be input to generate a correctly connected predicate“ broken. ”.

  With the process described above, the simplest predicate “broken” can be generated without changing the meaning of the predicate that represents the actual event that occurred, in response to the input “PC is broken”. . By using the method of the present invention, all the predicates that could not be put together by the method of Non-Patent Document 2 described in <Prior Art 2>, such as “It's broken” or “It's broken” Normalized to the form As a result, predicate expressions representing equivalent events necessary for mining can be grouped as “same” regardless of the surface layer form of the input sentence.

  Next, the operation when a more complicated sentence “It seems that the clock may have been broken” is input will be described. As in the case of the first embodiment, the morphological analysis unit 2 performs morpheme analysis on the input sentence (s1), and the morpheme analysis result shown in FIG. 11 is obtained. Part extraction is performed (s2), and the predicate extraction result shown in FIG. 12 is obtained.

  At this time, in the predicate extraction, “ta”, “may / know (ru) / not”, “ta”, “like”, and “ta” that match the entries in the function expression meaning label dictionary 1 are recognized as function words first. . Next, “no” and “da” belonging to the part of speech “non-independent category” and “auxiliary verb” are recognized as function words. Finally, “broken” belonging to the part of speech “verb—independence” is recognized as a content word. As a result, “It seems that it may have been broken” is extracted as a predicate.

<Semantic labeling>
Next, in the semantic label assigning section 4, semantic label assignment using the functional expression semantic label dictionary 1 is performed on the word information for each morpheme corresponding to the predicate of the input sentence, as in the case of the first embodiment. (S3).

  As mentioned above, if the predicate was "It seemed that it might have been broken", according to the longest match method from the back, "ta" is first analyzed and the meaning label "Complete" is given. Is done. Next, “da” and “no” are analyzed, but since there is no entry in the function expression semantic label dictionary 1, an empty semantic label “NULL” is given. Next, “like”, “ta”, “may / know (ru) / not”, and “ta” are analyzed, and the meaning labels “estimate”, “complete”, “estimate”, and “complete” are assigned, respectively. This semantic label assignment operation is terminated immediately before the content word “broken”, which is the last morpheme of the predicate. FIG. 13 shows an example of the meaning label assignment result in this embodiment.

<Function word deletion based on semantic labels>
<Deleting NULL>
Next, in the NULL deletion unit 5, the empty semantic label “NULL” is assigned to the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is added, as in the first embodiment. All function word entries (word information) are deleted (s4).

  As mentioned above, if the predicate's functional expression was “It seemed like it did not exist”, “no” and “da” with the empty semantic label “NULL” were deleted, and “ It may be simplified to a functional expression that it may not have been. FIG. 14 shows an example of a NULL deletion result in this embodiment.

<Delete redundant label>
Next, in the redundancy rule application unit 6, as in the case of the first embodiment, the function expression meaning label dictionary is applied to the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is added and after the NULL is deleted. Using the redundancy rule described in 1, all redundant function word entries (word information) are deleted (s5).

  Since the word information for each morpheme corresponding to the predicate “it may have been broken” in this example has three semantic labels “complete” and two “estimates”, the redundancy rule is applied. There is a need to. The redundancy rule of “complete” is “the first (if the surface layer shape is the same), the first (Last) is left otherwise”. On the other hand, the redundancy rule of “estimation” is “leave first”.

  Along with this, redundant semantic labels are reduced. In this embodiment, since “ta” indicating completion is the same in the surface layer shape, the first “ta” is retained. “Maybe” and “Like” representing the guess are retained along the redundancy rule. As a result, the function words “ta” “kamo” “know” “no” remain. FIG. 15 shows an example of the redundant label deletion result in this embodiment.

<Utilization generation>
Finally, in the utilization generation unit 7, the word information for each morpheme corresponding to the predicate of the input sentence after deleting the function word that does not affect the meaning of the predicate and the redundant function word of the first embodiment. Similarly to the case, each morpheme is connected to generate a predicate of the input statement (s6).

  In this embodiment, it is necessary to connect the verb “broken” to the functional expressions “ta”, “kam”, “know”, and “not”, but the content word “broken” is connected to the utilization generator based on the language model described above. "Denotation; part of speech; inflection type" is "broken; verb-independence; one step" and "ta; auxiliary verb; special-ta" of functional expression "ta", "kam; particle-auxiliary particle" of "kam" “Knowledge (verb)”, “Verb—independence; one step”, “No”, “No; auxiliary verb; Special / Nai”, and “.” In the morpheme “.” Representing the end of the sentence. ”Is generated, a correctly connected predicate“ may have been broken ”is generated.

  By the process described above, “It seems that it may have been broken” is normalized as “It may have been broken”, and it is normalized to the simplest form, but different from “It has broken” Normalized to show meaning. As a result, unlike the method of Non-Patent Document 1 described in <Prior Art 1>, normalization can be performed in which the difference in meaning represented by the predicate is clearly distinguished.

  1: functional expression meaning label dictionary, 2: morpheme analysis unit, 3: predicate extraction unit, 4: semantic label assignment unit, 5: NULL deletion unit, 6: redundancy rule application unit, 7: utilization generation unit.

Claims (7)

Without changing the meaning of the predicate of the input statement that consists of a combination of a content word and a functional expression consisting of a character string that follows it and contains one or more function words. To convert to a simple form,
A morpheme analyzer performs morpheme analysis on the input sentence and outputs word information for each morpheme;
A predicate extraction unit that extracts word information for each morpheme corresponding to a content word and a functional expression subsequent to the content word among word information for each morpheme of the input sentence as a predicate of the input sentence;
The semantic label assignment unit uses a functional expression semantic label dictionary that combines at least a semantic label that represents the meaning of the functional expression that affects the meaning of the predicate and a list of functional expressions corresponding to each semantic label. Of the word information for each morpheme corresponding to the predicate of the sentence, the step of giving a semantic label to the word information of the function word string constituting the function expression;
The function word deletion unit deletes the word information of the function word that does not affect the meaning of the predicate and the word information of the redundant function word from the word information for each morpheme corresponding to the predicate of the input sentence after the meaning label is given. And steps to
Utilization generation unit generates predicate of input sentence based on word information for each morpheme corresponding to predicate of input sentence after deleting function words that do not affect meaning of previous description part and redundant function words And a predicate function expression normalization method. Function word deletion step
A NULL deletion unit that deletes word information of a functional word to which no semantic label has been assigned, out of word information for each morpheme corresponding to the predicate of the input sentence after the semantic label has been assigned;
The redundancy rule application unit uses a redundancy rule for each semantic label that indicates how to retain these function words when the function expression corresponding to one predicate includes a plurality of function words with the same semantic label. And deleting the word information of the functional word excluded by the redundancy rule from the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is added. Description predicate function expression normalization method. Semantic labels that characterize the tense of events represented by predicates, semantic labels and predicates for distinguishing negation or affirmation of events represented by predicates, as semantic labels that represent the meaning of functional expressions that affect the meaning of predicates The predicate function expression normalization method according to claim 1 or 2, wherein a semantic label for distinguishing whether or not the subject's subjectivity is included in the event to be expressed is used. Without changing the meaning of the predicate of the input statement that consists of a combination of a content word and a functional expression consisting of a character string that follows it and contains one or more function words. It is a device that converts to a simple form,
A function expression meaning label dictionary comprising at least a combination of a meaning label that represents the meaning of the function expression that affects the meaning of the predicate, and a list of function expressions corresponding to each meaning label;
A morpheme analysis unit that performs morpheme analysis processing on the input sentence and outputs word information for each morpheme,
Among the word information for each morpheme of the input sentence, a predicate extraction unit that extracts word information for each morpheme corresponding to the content word and the functional expression subsequent thereto as a predicate of the input sentence;
Using a functional expression meaning label dictionary, among the word information for each morpheme corresponding to the predicate of the input sentence, a semantic label giving unit for giving a semantic label to the word information of the functional word string constituting the functional expression;
A function word deleting unit that deletes word information of redundant function words and function words that do not affect the meaning of the predicate among the word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is given; ,
A utilization generator that generates a predicate of the input sentence based on word information for each morpheme corresponding to the predicate of the input sentence after the function word that does not affect the meaning of the previous description part and the redundant function word are deleted; A predicate function expression normalizing apparatus comprising: The function word deletion part
A NULL deletion unit that deletes word information of a function word to which no semantic label is assigned, out of word information for each morpheme corresponding to the predicate of the input sentence after the semantic label is assigned
If the functional expression corresponding to one predicate contains multiple functional words with the same semantic label, the redundant rule for each semantic label that indicates how to retain those functional words is used. 5. The predicate according to claim 4, further comprising: a redundancy rule application unit that deletes word information of function words excluded by the redundancy rule from word information for each morpheme corresponding to the predicate of the input sentence. Functional expression normalization device. Semantic labels that characterize the tense of events represented by predicates, semantic labels and predicates for distinguishing negation or affirmation of events represented by predicates, as semantic labels that represent the meaning of functional expressions that affect the meaning of predicates The predicate function expression normalization apparatus according to claim 4 or 5, wherein a semantic label for distinguishing whether or not the subject's subjectivity is included in the represented event is used.   The program for functioning a computer as each means of the apparatus in any one of Claims 4 thru | or 6.

Images