JP2009176168A - Language processor, language processing method, language processing program, and recording medium recording same program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an essential surface case through the use of a case conversion rule with an appropriate conversion rule described therein. <P>SOLUTION: A language processor includes a case conversion rule table 2 for storing a rule to convert a modification state between a predicate or an action noun and a word other than it or a word attribute into case relation between the predicate or the action noun and the word other than it. A text input by a case conversion part 3 is output after being converted into the argument structure of the predicate and the action noun through the use of the modification state of a text and the rule of the case conversion rule table 2. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a language analysis used in a question answering system in which a computer answers a question expressed in a natural language, an information retrieval system, an information extraction system, an automatic summarization system, an automatic translation system, an automatic paraphrase system, a speech recognition system, etc. The present invention relates to an apparatus, a language processing method, a language processing program, and a recording medium on which the language processing program is recorded.

  In a conventional language processing device, a probabilistic model is assumed for the case frame of the predicate, machine learning is performed using corpus data in which the correct answer of the case frame is manually assigned, and the parameters of the probability model are estimated and determined. An apparatus that outputs the most likely term structure using a probabilistic model has been proposed (see, for example, Non-Patent Document 1). In this method, in a sentence, given a word that is a predicate and a word that is a term for the predicate, the case frame that expresses the semantic attribute of the word is expressed in information theory. It is a method to check whether the rule is powerful, and does not deal with predicate and term recognition, anaphora analysis, and modifier determination.

In a conventional language processing device that outputs a predicate term structure, a verb that is registered in advance in a dictionary can only perform analysis using case information given to verbs, adjectives, and nouns. When noun usage was used, it could not be analyzed properly. Even if case information is registered in the dictionary, if there are multiple verbs and noun usages, there is no clear standard on which usage case information is used for analysis, and there is no need to rely on human resources. Adjustment is necessary, and the adjustment is very labor intensive, and it is difficult to find an adjustment method that improves the analysis accuracy by the adjustment. Therefore, in the method proposed in Non-Patent Document 2, a method for automatically constructing a probabilistic model of a predicate term structure from a large-scale text corpus has been proposed.
Lee Wang, Naoki Abe, “Learning Dependencies Between Case Slots”, Information Processing Society of Japan Research Report, Natural Language Processing Study Group Report, Vol. 96, no. 114, PP. 93-99. Daisuke Kawahara and Ikuo Kurohashi, “Integrated probabilistic model of syntactic and case analysis based on large-scale case frames acquired from the Web”, Proc. Of the 13th Annual Conference of the Language Processing Society, 2006.

  However, in the question answering, information retrieval system, etc., the predicate “what is what” and “what is what”, and the essential surface case for the basic form of the action noun are useful information. In this method, only the surface case for the surface form of the predicate is dealt with. Cannot output. In addition, when the required surface case is omitted, the required surface case cannot be output. Furthermore, there is a problem that the required surface case modifiers are not considered in terms of how far they are included in the term, and the output of the required surface case including appropriate modifiers has not necessarily been realized.

  The object of the present invention is to solve the above-mentioned problems, and for the predicates and behavioral nouns, the dependency relationship between the predicates or behavioral nouns, their part of speech, their prescriptive meaning attributes, and the semantic headings, and the semantics. Using the case conversion rules that describe appropriate conversion rules to essential surface cases from the features and adjuncts of the part of speech, semantic category, etc., the required surface case is obtained and even if the case conversion rules are applied If the case is not obtained, the required surface case is determined based on the co-occurrence information of the predicate or behavioral noun and the term and the topic level, and how many modifiers are included for the obtained surface case word An object of the present invention is to provide a language processing apparatus, a language processing method, a language processing program, and a recording medium on which a language processing program is recorded.

  In order to achieve the above object, the invention according to claim 1 is a language processing device that outputs a term structure for a predicate or a behavioral noun in an input text. Case conversion rule storage means for storing a rule for converting a dependency state between words or word attributes into a case relationship between a predicate or a behavioral noun and other words, and a dependency state of text and the case conversion rule storage means And a case conversion means for converting the input text into a predicate and a term structure of a behavioral noun by applying a rule for conversion into a case relationship.

  The invention according to claim 2 is characterized in that, in claim 1, the rule for converting to the case relation is generated by machine learning from a text given a correct case relation and a dependency state. .

  The invention according to claim 3 is characterized in that, in claim 1 or 2, the dependency state is information of a relationship original character and a relationship character for a semantic main character of a phrase.

  The invention according to claim 4 is the invention according to claim 1 or 2, wherein the dependency state is information about the semantic main word of the phrase, the information of the auxiliary word, the predecessor main word, and the semantic main word, or the semantic main word. It is characterized by the meaning category or predicate or behavioral noun prescriptive meaning attribute or modifier phrase of the part of speech or semantic main word or the genre or category information to which the word or text appears in the same text.

The invention according to claim 5 is the invention according to claim 1, wherein the rule for converting to the case relation is a syntax of a text to be analyzed described in a natural language in which a term structure is given to a predicate or an action noun. A syntactic analysis unit that analyzes and analyzes a set of clauses and a dependency relationship between clauses, and identifies a semantic main word for each phrase from words determined to be independent words in the phrase by the syntactic analysis unit; A main part specifying unit for creating the main part and a dependency part of the main part for each main part of the main part table, and a subordinate dependency for adding an additional word following the main part to the main part table in the clause to which the main part belongs From the analysis section and the term structure of the predicate or behavioral noun assigned to the text, the required superficial case for the basic form of the predicate and behavioral noun and the phrase to which each word belongs are extracted, and the term structure correct A table is created, and each clause of the term structure correct answer table is converted to a head with reference to the head table, and the dependency relationship of clauses in the input text corresponding to the required case of each predicate and action noun is A case conversion rule learning unit that adds a term structure correct answer table and learns case conversion rules using the term structure correct answer table and a case conversion rule output unit that outputs the obtained predicate and action noun as case conversion rules It is generated.

  The invention according to claim 6 further includes a thesaurus according to claim 5, and the case conversion rule learning unit searches the thesaurus to extract a superordinate concept of words in the term structure correct answer table. A rule in which the extracted word and the original word are replaced is further created and added to the term structure correct answer table.

  The invention according to claim 7 is that in claim 5, the case conversion rule learning unit learns the case conversion rule by machine learning of the correct case conversion rule described in the item structure correct answer table. Features.

  The invention according to claim 8 is a language processing method for outputting a term structure for a predicate or an action noun in an input text, the step of inputting the text and the dependency state of the text, Applying a case conversion rule stored in the case conversion rule storage means to the predicate or behavioral noun according to the dependency state of the text, and converting to a term structure by the case conversion means; and Outputting a term structure for the obtained predicate or action noun.

  The invention according to claim 9 is characterized in that, in claim 8, the case conversion rule is generated by machine learning from a text having a correct case relation and a dependency state.

  The invention according to claim 10 is characterized in that, in claim 8 or 9, the dependency state is information of a relationship original character and a relationship character for a semantic main character of a phrase.

  The invention according to claim 11 is the invention according to claim 8 or 9, wherein the dependency state is the information of the ancillary words or the semantic main words for the semantic main words of the clause, the original characters of the original characters, the dependent characters and the semantic characters. It is characterized by the meaning category or predicate or behavioral noun prescriptive meaning attribute or modifier phrase of the part of speech or semantic main word or the genre or category information to which the word or text appears in the same text.

The invention according to claim 12 is the text to be analyzed according to claim 8, in which the case conversion rule is a text to be analyzed in which the syntax analysis unit is described in a natural language in which a term structure is given to a predicate or an action noun And analyzing the dependency relationship between the phrase set and the phrase, and the main word specifying unit determines a semantic main word for each phrase from the words determined by the syntactic analysis unit as independent words in the phrase. The step of identifying and creating a head table, and the dependency analysis unit between the heads adds a dependency clause of the clause for each clause of the head table, and for the head, the ancillary word following in the clause to which the head belongs And a case conversion rule learning unit from the term structure of the predicate or behavioral noun assigned to the text, the required surface case for the basic form of the predicate and behavioral noun Then, the clauses to which each word belongs are extracted, a term structure correct answer table is created, each clause of the term structure correct answer table is converted to a main word with reference to the main word table, and the required case of each predicate and action noun Are generated from the step of adding the dependency relations of clauses in the input text corresponding to the item structure correct answer table and the case conversion rule output unit outputting the obtained predicate and action noun as case conversion rules. It is characterized by that.

  The invention according to claim 13 is the invention according to claim 12, further comprising a thesaurus, wherein the case conversion rule learning unit searches the thesaurus to extract a superordinate concept of words in the term structure correct answer table. Later, a rule in which the extracted word and the original word are replaced is further created and added to the term structure correct answer table.

  Further, in the invention according to claim 14, in claim 12, the case conversion rule learning unit learns the case conversion rule by machine learning of the correct case conversion rule described in the item structure correct answer table. Features.

  The invention according to claim 15 is a language processing program that causes a computer to function as each means in the language processing apparatus according to any one of claims 1 to 7.

  The invention according to claim 16 is a computer-readable recording medium on which the language processing program according to claim 15 is recorded.

  According to the present invention, there is an advantage that the term structure of the basic form of the predicate or the action noun can be output with high accuracy for the text expressed in the natural language.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the basic system configuration of the language processing apparatus according to this embodiment. The language processing apparatus 1 stores a case conversion rule table storing rules for converting a dependency state between words or word attributes other than predicate or behavioral noun fear into a case relationship between a predicate or behavioral noun and other words. 2 and a case conversion unit 3 which converts the input text into a predicate and a term structure of an action noun using the text dependency state and the case conversion rule.

  The principle operation of the system shown in FIG. 1 will be described below with reference to the flowchart shown in FIG.

  First, the user inputs text and a text dependency state to the language processing apparatus 1 (step 1). Here, the dependency state of the text can be given manually or the output of a program such as a parser can be given.

  Next, the case conversion rules stored in the case conversion rule table 2 are applied to the predicates or behavioral nouns in the text according to the dependency state of the text, and the case conversion unit 3 converts the case into a term structure. (Step 2). The case conversion rule table 2 can be given manually in advance, or can be automatically created and given by a program that implements a technique such as machine learning.

  Finally, the term structure for the predicate or the action noun obtained in step 2 is output from the case conversion unit 3 to end the operation (step 3).

  FIG. 3 is a block diagram showing the basic system configuration of the language processing apparatus 10 according to the present embodiment.

  The language processing apparatus 10 includes a syntax analysis unit 101, a proper noun linking unit 102, a subject specifying unit 103, an inter-subject dependency analysis unit 104, a non-subject predicate table 124, a non-subject predicate storage unit 125, a case conversion unit 126, an anaphoric response. It has an analysis unit 127, a modifier phrase determination unit 128, a term structure output unit 110, a syntax analysis result table 120, a main table 121, a thesaurus 122, a case conversion rule table 106, a predicate term table 129, and a term structure table 130.

  Next, the operation of a specific language processing apparatus to which the system configuration of this embodiment shown in FIG. 3 is applied will be described with reference to the flowchart shown in FIG.

  First, an analysis target text written in a natural language is input to the language processing apparatus 10 (step 1).

  Next, the syntax analysis unit 101 parses the text, analyzes the clause set and the dependency relationship between clauses, and stores the analysis result in the syntax analysis result table 120 (step 2).

  For example, if you parse a sentence that says, “Mr. Tomiichi Murayama ate a delicious curry made by his wife”, as shown in FIG. , Get the phrase number to which the word belongs.

  Here, the phrase number is an integer greater than or equal to “0”, and is assigned “0, 1, 2,...” In order from the first phrase of the text. The proper noun tag is a tag given when a word is analyzed to be a proper noun or a part of a proper noun. For example, B-PERSON represents the first word of a proper noun of a person name, -PERSON represents a word that is not the head of a proper noun of a person name, and "O" represents a noun that is not a proper noun.

Further, as shown in FIG. 6, the syntax analysis unit 101 outputs a set of a clause number and a related clause number that is the number of the clause related to the clause. Here, when there is no destination at the end of the sentence, the destination clause number is set to “−1”. This output result is stored in the syntax analysis result table 120. In the syntax analysis unit 101, a single syntax analyzer such as CaboCha (reference document: Non-Patent Document 3) can be used.
Taku Kudo, Yuji Matsumoto, “Japanese Dependency Analysis by Chunking Stage Application”, IPSJ Journal, Vol. 43, no. 6, pp. 1834-1842, 2002.

  Next, the proper noun linking unit 102 looks at the proper noun tab fields of the syntax analysis result table 120 in order from the top, and collects one proper noun into one (step 3). For example, when the syntax analysis result table 120 is as shown in FIG. 5, the words “Murayama” and the word “Tomiichi” have proper noun tags “B-PERSON” and “I-PERSON”, respectively. The proper noun linking unit 102 concatenates these two words, and sets the word column as “Murayama Tomi City” and the word basic form column as “Murayama Tomi City”. The part-of-speech column is collectively referred to as “noun-proprietary noun-person name”, and the proper-noun tag column is also collectively referred to as “PERSON”. Also, word numbers are assigned in order from the first word of the text. The syntax analysis result table 120 is updated with the result. An example of the updated syntax analysis result table 120 is shown in FIG.

  Next, the main part specifying unit 103 specifies the semantic main part of each phrase (step 4). In each clause, the last word in the clause among the independent words in the clause, that is, the words whose part of speech is determined to be independent words in nouns and in verbs and adjectives Is extracted as a semantic head and is combined with the word number of the word and the phrase number of the phrase to which the word belongs to create a head table 121. An example of the main table 121 is shown in FIG.

  Next, the inter-subject dependency analysis unit 104 searches for the clause number of the clause number in the dependency clause number table with respect to the clause number of the subject table 121 and adds the dependency clause number to the subject table 121 ( Step 5). An example of the resulting main table 121 is shown in FIG.

  The inter-subject dependency analysis unit 104 adds, for each main word, the entire non-independent word that follows in the phrase to which each main word belongs to the main word table 121 as an attached word (step 6). However, the ones excluding the punctuation at the end are attached words. An example of the main table 121 as a result is shown in FIG.

  Next, in step 7, the non-subject predicate storage unit 125 stores in the non-subject predicate table 124 a list of predicates or behavioral nouns other than the main predicate in the clause. For example, in step 1, it is assumed that an analysis target sentence “the players ate the winning memorial cake made by the wife of the director” was input. At this time, the syntax analysis result in step 2 is as shown in FIG. 11, and the syntax analysis unit 101, as shown in FIG. 12, shows the clause number and the related clause that is the number of the clause to which the clause relates. Suppose you are outputting a set of numbers. Further, it is assumed that the main word table 121 created in step 6 is updated as shown in FIG.

  If there is a predicate or action noun that precedes the main word in each clause, it is stored in that order. For example, before the main word “cake” of the phrase number 2, there are operational nouns “winning” and “commemorative”, and are stored in the non-main word predicate table 124 in that order as shown in FIG. If there is no predicate or action noun in the clause before the main word “Make” as in clause number 1, it is stored as “−”.

  Next, the following processing is repeated for each main word in order from the front in the sentence (step 8). Based on the main table of FIG. 13, it is checked whether there is a destination to which the main word is related, or whether it is related from another word in the sentence. If there is a dependency relationship in step 9 (Y ), The case conversion rule is applied by the case conversion unit 126 (step 10). In step 9, if there is no dependency relationship (N), the anaphora analysis unit 127 performs anaphora analysis (step 11).

  Application of the case conversion rule in step 10 will be described. First, among the main words in the main word table of FIG. 13, for the predicate or the action noun, the other main words in the dependency relationship, their semantic categories, their word numbers, and the main words of the clause containing the main word A predicate term table 129 is created by assigning a prescriptive meaning attribute to the following ancillary word and phrase number, predicate or action noun using the main table 121 (FIG. 13), thesaurus 122, and the prescriptive meaning attribute system. For example, FIG. 15 shows an example of the predicate term table 129 obtained from the table shown in FIG. Using FIG. 15, an optimal conversion rule is selected for each different main word.

  For example, as shown in FIG. 16, when the weight of conversion from the relationship between the word or the semantic category and the required surface case is obtained as a dictionary for the predicate or the action noun itself, it appears in the predicate term table 129. When a combination of conversion rules is applied to a word and a dependency state, a combination of conversion rules that obtains an evaluation value having the largest weight evaluation value (for example, a linear sum of weights) is applied.

  If the rule for the predicate or behavioral noun itself as shown in FIG. 16 is not applied, the conversion of the predicate or behavioral noun to the meaning attribute of the predicate or behavioral noun is considered (FIG. 17). Similarly, an evaluation value is calculated using the given evaluation function, and a combination of conversion rules that gives the highest evaluation value is applied.

  As a result, if conversion is not applied, a conversion rule for predicates or general behavioral nouns is considered (FIG. 17). Similarly, an evaluation value is calculated using the given evaluation function, and a combination of conversion rules that gives the highest evaluation value is applied. As a result, in this reference example, it is assumed that a term structure table 130 as shown in FIG. 18 is obtained. Thereafter, the anaphoric analysis unit 127 performs anaphoric analysis, and the process proceeds to step 11.

  In step 11, with respect to the predicate or the behavioral noun, the anaphora analysis is performed on the part where the term structure has not been determined by the processing so far, and the term structure is determined. “Director”, “winner” and “commemorative” which are predicates or behavioral nouns stored in the non-subject predicate table and which have no dependency relationship are also added to the item structure table 130 shown in FIG. Suppose that it is updated as shown in FIG. Here, the anaphora analysis unit 127 performs anaphora analysis on the portion of the ga-case, wo-case, and d-case where the word is not filled in, and the anaphora destination is determined. Fill with words.

  Various methods can be considered for the method of anaphora analysis. Here, anaphora analysis is performed using the word having the highest “topic” among the words appearing in the sentence and in the same context. Here, topic level refers to the words that appear even when summarizing the entire sentence at various summarization rates from a syntactical viewpoint, semantic viewpoint, word appearance frequency, etc. The high score is scored so that only a summary with a low summarization rate gives a low score to words that do not appear.

  In addition, when a word is not filled in the processing so far, “exog” representing “external world response” is filled when it is assumed that the case is always an indispensable case. As a result, the term structure table 130 is as shown in FIG.

  Next, an output modifier is specified by the modifier determining unit 128 (step 12). For the main word obtained in the item structure table 130 of FIG. 20, it is determined whether to output the main letter as it is or to add a modifier. By comparing the frequency of appearance of the main word as it is and the frequency of appearance of the word group when the modifier is added, the word group is recognized as a word that outputs a word group having a high frequency form.

  For example, when the word “players” from the main word “players” appears more frequently in a general corpus, “players” are replaced with “players” and output. As the candidates of these word groups, using the result of the syntax analysis in FIG. 11, consecutive words in the same phrase are added to the main word, and the frequency is calculated. Further, the column of word numbers and phrase numbers is deleted, and the item structure table is updated as shown in FIG.

  The finally obtained term structure table is outputted from the term structure output unit 110 as a predicate and a behavioral noun term structure analysis result (step 13).

  In this embodiment, an example in which the term structure to be output is a term structure related to an essential surface case has been described. However, even if the output term structure is related to an arbitrary surface case or a deep case, the case conversion rule table is displayed. It can be realized by the same means as long as it corresponds to them.

  In addition, even when the processing target is a foreign language text such as English, relations in phrase structures such as noun phrases / verb phrases composed of parts of speech other than verbs / nouns or words instead of dependency-related adjuncts By using a conversion rule that is used as a relationship, an analysis result can be obtained using this processing method.

  FIG. 23 is a block diagram showing a principle system configuration of a dictionary creation device used in the language processing device. The dictionary creation device 11 includes a case conversion rule learning unit 12.

  Next, the principle operation of the system configuration shown in FIG. 23 will be described with reference to the flowchart shown in FIG.

  First, the user inputs the text and the correct term structure of the text into the case conversion rule learning unit 12 to the dictionary creation device 11 (step 1).

  Next, a case conversion rule for converting the dependency state between the predicate or behavioral noun in the text and the main word of each clause in the text into a predicate term structure is created using machine learning technology (step 2). Finally, the case conversion rule obtained in step 2 is output from the case conversion rule learning unit 12 and the operation is terminated (step 3).

  FIG. 25 is a block diagram showing a specific system configuration of the dictionary creating apparatus 100 of the dictionary creating apparatus 11, in which the same parts as those of the language processing apparatus shown in FIG. The dictionary creation apparatus 100 includes a syntax analysis unit 101, a proper noun linking unit 102, a subject specifying unit 103, a dependency dependency analysis unit 104, a case conversion rule learning unit 105, a syntax analysis result table 120, a subject table 121, and a thesaurus 122. A term structure correct answer table 123 and a case conversion rule output unit 107.

  Hereinafter, the operation of the specific dictionary creation apparatus 100 shown in FIG. 25 will be described with reference to the flowchart shown in FIG.

  First, training text written in a natural language is input to the syntax analysis unit 101 to the dictionary creation device 100 (step 1).

  Next, the syntax analysis unit 101 parses the text to identify a dependency set between clause sets and clauses, and stores the analysis result in the syntax analysis result table 120 (step 2). For example, when the sentence “Mr. Tomiichi Murayama ate a delicious curry made by his wife” was parsed, words, basic word forms, parts of speech, proper noun tags in order from the top of the text as shown in FIG. , Get the phrase number to which the word belongs. Here, the phrase number is an integer greater than or equal to “0”, and is assigned as 0, 1, 2,... In order from the first phrase of the text.

  The proper noun tag is a tag given when a word is analyzed as a proper noun or a part of a proper noun. For example, B-PERSON represents the first word of a proper noun of a person name, I-PERSON represents a word that is not the head of a proper noun of a person name, and “O” represents a noun that is not a proper noun.

  Further, as shown in FIG. 6, the syntax analysis unit 101 outputs a set of a clause number and a related clause number which is the number of the clause related to the clause. Here, when there is no destination at the end of the sentence, the destination clause number is set to “−1”. This output result is stored in the syntax analysis result table 120. In the syntax analysis unit 101, a single syntax analyzer such as CaboCha (reference document: Non-Patent Document 3) can be used.

  Next, the proper noun linking unit 102 looks at the proper noun tab column of the syntax analysis result table 120 in order from the top, combines one proper noun into one word, and assigns word numbers in order from the beginning of the text. To do. Here, the word number is an integer greater than or equal to “0”, and in order from the first word of the text, 0, 1, 2,. . . (Step 3). For example, when the syntax analysis result table 120 is as shown in FIG. 5, the words “Murayama” and the word “Fumiichi” have proper noun tags “B-PERSON” and “I-PERSON”, respectively. The proper noun linking unit 102 concatenates these two words, and sets the word column as “Murayama Tomi City” and the word basic form column as “Murayama Tomi City”. The part-of-speech column is collectively referred to as “noun-proprietary noun-person name”, and the proper-noun tag column is also collectively referred to as “PERSON”. Also, word numbers are assigned in order from the first word of the text. The syntax analysis result table 120 is updated with the result. An example of the updated syntax analysis result table 120 is shown in FIG.

  Next, the main part specifying unit 103 specifies the semantic main part of each phrase (step 4). In each clause, the last word in the clause among the independent words in the clause, that is, the words whose part of speech is determined to be independent words in nouns and in verbs and adjectives Is extracted as a semantic main word, and the main word table 121 is created together with the word number of the word and the phrase number of the phrase to which the word belongs. An example of the main table 121 is shown in FIG.

  Next, the dependency dependency analysis unit 104 acquires the clause number of the dependency destination for each clause number in the clause number column of the subject table 121 using the dependency clause number for the clause number described in the dependency clause number table. Then, the related phrase number is added to the main word table 121 (step 5). FIG. 9 shows an example of the main word table 121 obtained by using the relation clause number table shown in FIG. 6 and the main word table 121 shown in FIG.

  The inter-subject dependency analysis unit 104 adds, for each main word, the entire non-independent word that follows in the phrase to which each main word belongs to the main word table 121 as an attached word (step 6). However, if there is a punctuation mark at the end of the subsequent non-independent words, the word excluding the punctuation mark is used as an appendix. An example of the main table 121 as a result is shown in FIG.

  Next, in step 7, the case conversion rule learning unit 105 learns case conversion rules. As the data for learning, the correct term structure for the training text "Prime Minister Tomiichi Murayama ate a delicious curry made by his wife", which was manually assigned to the required superficial case of the basic form of predicate or action noun. Is given as shown in FIG.

  Here, a portion surrounded by tags <NP ID = number> and </ NP> represents a noun phrase, and a portion surrounded by <PRED ~> and </ PRED> represents a predicate. Although it does not appear in this text, a behavioral noun is assumed to be enclosed in <EVENT-> and </ EVENT> tags. In addition, the description such as “ga =“ 2 ”wo =“ 3 ”” described in the “˜” part of the <PRED ~> tag and the <EVENT ~> tag is “the basic form of this predicate or action noun”. On the other hand, the correct answer of a term that takes a case is a noun phrase whose ID number is 2, and the correct answer of a term that takes a case is a noun phrase whose ID number is 3. For this data, the case conversion rule learning unit 125 uses the training text, the correct term structure, and the syntax analysis result table 120 to use a predicate sandwiched between <PRED ~> tag and </ PRED> tag and <EVENT For the behavioral nouns enclosed by the ~> tag and the </ EVENT> tag, the term structure correct answer table 123 is created which stores the essential surface case for the basic form and the phrase number to which each word belongs. An example of the term structure correct answer table 123 is shown in FIG. Here, for the sake of simplicity, it is assumed that only the mandatory case cases are treated as ga, wo, and d, but other cases can be processed in the same manner.

  Next, the case conversion rule learning unit 105 replaces each word in the term structure correct answer table 123 with the main word for the phrase number. The word number corresponding to the main word is also stored. If the main word is not a partial character string of the word before replacement, the word field and phrase number are replaced with "-".

  If the predicate or behavioral noun itself is not the main word of the clause, the predicate or behavioral noun is deleted from the term structure correct answer table 123. FIG. 29 shows an example of the term structure correct answer table 123 after being updated in this way.

  Next, the case conversion rule learning unit 105 adds the dependency relationship information in the original text to the term structure correct answer table 123 using the information in the main table 121 as shown in FIG. An example of the term structure correct answer table 123 after being updated in this way is shown in FIG. Here, the relationship is related to the predicate or behavioral noun if the term depends on the predicate or behavioral noun, and if the predicate or behavioral noun depends on the term, “( The reverse) "is added.

Next, the case conversion rule learning unit 105 assigns a semantic category to the word using a thesaurus 122 such as a Japanese vocabulary system (reference document: non-patent document 4). The word number and phrase number are deleted. The word column of the predicate or action noun is converted into the basic form of the word using the syntax analysis result table 120.
Satoru Ikehara, Masahiro Miyazaki, Atsushi Shirai, Akio Yokoo, Hiroaki Nakaiwa, Kentaro Ogura, Yoshifumi Oyama, Yoshihiko Hayashi, Japanese Vocabulary System. Iwanami Shoten (1997). As a result, an example of the updated term structure correct answer table 123 is shown in FIG. A large number of such tables are collected, and a case conversion rule that is generally established is learned using a machine learning method. For example, for simplicity, when there is a term structure correct answer table 123 having only five items as shown in FIG. 32, the operation of the case conversion rule learning unit 105 for this table will be described. Here, case conversion rules are learned for each predicate or action noun.

  First, the case conversion rule learning unit 105 generates training data dedicated to the predicate “create” for the first to third lines in which the “predicate or action noun” column of the term structure correct answer table 123 is “create”. To do.

  The case conversion rule learning unit 105 extracts the structure before the case conversion from the first to third lines of the term structure table 123 and uses it as the pre-conversion feature used for machine learning. For example, pre-conversion feature 1 is “word: wife, relationship: no”, pre-conversion feature 2 is “word: curry, relationship: ta (reverse)”, and pre-conversion feature 3 is “word: me, relationship: The pre-conversion feature 4 is “word: stew, relationship: ta (reverse)”, the pre-conversion feature 5 is “word: company, relationship: g”, and the pre-conversion feature 6 is “word: rule, relationship: (Reverse) ”. As a pre-conversion feature, a feature such as “semantic category: female, relationship: no” using a semantic category or “semantic category: entity, relationship: no” using a semantic category higher than the semantic category is used. However, in order to simplify the explanation, it is assumed that the above six features are used.

Next, the case conversion rule learning unit 105 creates pre-conversion structure vectors x _i (i = 1 to 3) for the first to third lines of the term structure correct answer table 123. Here, the pre-conversion vector is a vector having elements of 1 to 6 when the feature is described in the term structure correct answer table and 0 when the feature is not described. For example, since only the feature 1 and the feature 2 are described in the first row of the term structure correct answer table 123, x ₁ = ( ₁ , ₁ , 0, 0, 0, 0) t (t represents transposition) , The same applies hereinafter), and only the feature 3 and the feature 4 are described in the second row of the term structure correct answer table 123. Therefore, x ₂ = (0,0,1,1,0,0) t Is generated. Similarly, x ₃ is also generated.

  Next, the case conversion rule learning unit 105 extracts the structure after the case conversion from the first to third lines of the term structure correct answer table 123 and uses it as the post-conversion feature used for machine learning. For example, if the post-conversion feature 1 is “first appearing word” and the post-conversion feature 2 is “second appearing word”, and if these words are converted, for example, the surface case for the basic form takes a case of 1, The features are defined such as 2 for the case, 3 for the second case, and 4 for the other cases.

Next, the case conversion rule learning unit 105 creates post-conversion structure vectors y _i (i = 1 to 3) for the first to third lines of the term structure correct answer table 123. For example, for the first line of the term structure correct answer table 123, the first word “wife” is converted to ga and the second word “curry” is converted to wo, so y ₁ = (1,2) t is generated. Similarly, y ₂ = (1,2) t and y ₃ = (1,2) t are generated for the second and third lines of the term structure correct answer table 123.

Hereinafter, the conversion from the pre-conversion structure vector x _i to the post-conversion structure vector y _i is referred to as “case conversion rule i”.

Next, the case conversion rule learning unit 105 performs statistical machine learning on these case conversion rules, and appropriately weights the case conversion rules. As an example, a method called SVMstruct, and a method of learning using a machine learning method called SVMΔs ₂ (Δs is a superscript) in Non-Patent Document 5 shown below will be described below. .
Ioannis Tsochantaridis, Thomas Hofmann, Thorsten Joachims, Yasemin Altun, "Support Vector Machine Learning for Interdependent and Structured Output Spaces", The Proceedings of The Twenty-First, International Conference on Machine Learning (ICML2004) SVM △ s2 (Δs superscript) Then, the weight vector w and the slack variable vector ξ are calculated so as to minimize the following equation 2 while satisfying the inequality represented by the following equation 1.

Here, i is the same index number as _i in the above x _i and y _i , x _i is the same pre-conversion structure vector as above, and y _i is the same correct post-conversion structure vector as above. Y is a whole set of post-conversion structure vectors that can be obtained when the pre-conversion structure vector x _i is transformed, and y ^to _{i, k} are correct ones of the post-conversion structure vectors that can be taken. A structure of incorrect answers different from the structure y _i , which represents the k-th structure (k is an index number). δψ (x _i , y ^to _{i, k} ) is a difference between ψ (x _i , y _i ) and ψ (x _i , y ^to _{i, k} ) defined by the following equation (3). ψ (x _i , y _i ) is a vector generated from the pre-conversion vector x _i and the post-conversion vector y _i , and the logical product of the feature represented by the element of x _{i and} the feature represented by the element of y _i is an element. Is a vector. FIG. 20 shows an example of the vector ψ (x ₁ , y ₁ ) with respect to x ₁ and y ₁ as an example of ψ (x _i , y _i ). ξ _i is the i th element of the slack variable vector ξ and is a slack variable for the structure vector x _i before conversion, and is a variable having a real value equal to or greater than “0”. Δ (y _i , y ^˜ _{i, k} ) is a loss function that represents how different the incorrect solution structure y ^˜ _{i, k} is from the correct post-conversion structure y _i . , And the number of case elements that do not match between the structures y _i and y ^˜ _{i, k} . C is a parameter representing a trade-off between identification error for the entire training data and margin maximization, and is a positive real value given by the user. n is the maximum value of the index number i, that is, the number of training data, and <A, B> represents the inner product of the vector A and the vector B.

SVMΔs ₂ (Δs is a superscript) solves an optimization problem that minimizes the following equation 2 while satisfying the inequality represented by equation 1, but does not solve this optimization problem directly, but a dual problem. This optimization problem is solved after being converted into the following equation (4).

Here, α _{i, k} is a Lagrangian coefficient corresponding to y ^to _{i, k} . Further, δii ′ is a function that takes 1 when i = i ′ and takes 0 when i ≠ i ′. In addition, in the vector inner product portion in Equation 4, a kernel function K can be defined and used instead of the inner product as shown in Equation 5 below.

In the present embodiment, a second-order polynomial kernel K (A, B) = (<A, B> +1) ² is used as the kernel function. Here, A and B are vectors of the same dimension.

Similarly, the case conversion rule learning unit 105 performs training data dedicated to the predicate “eat” on the fourth to fifth lines in which the “predicate or behavioral noun” column of the term structure correct answer table 123 is “eat”. , And learning weights by machine learning for the generated case conversion rules, and appropriately weighting the case conversion rules. In this example, case conversion rules are generated and weights are learned only for each predicate or action noun, but case conversion is also performed for each predicate attribute level and all predicates or action noun levels. Generate rules and learn weights. Here, the term meaning attribute refers to verbs, adjectives, etc., and attributes such as “physical movement”, “body movement”, and “emotional movement” are manually set based on the meaning. The word meaning attribute system "(reference document: non-patent document 6) can be given.
Hiroaki Nakaiwa and Satoru Ikehara, “Classification of Japanese semantic meanings focusing on syntactic correspondence between Japanese and English”, Transactions of Information Processing Society of Japan, Vol. 38, no. 2, pp. 215-225, 1997.

  An example of case conversion rules and weights obtained by machine learning is shown in FIG.

  The finally obtained table is output as an analysis result from the case conversion rule output unit 107 to obtain a case conversion rule table (dictionary) (step 8).

  In this specific example, a specific example in which the term structure to be output is a term structure related to the required surface case has been described. However, even if the output term structure is related to any surface case or deep case, the correct answer term to be input is input. The structure can be realized by the same means as long as the structure is adapted to them.

  Also, a learning method different from the machine learning method described above can be used as long as the learning means can learn the weight of the case conversion rule.

  In addition, even in the case of foreign language texts such as English, the relationship in phrase structures such as noun phrases / verb phrases composed of parts of speech other than verbs / nouns and words, instead of dependency-related adjuncts. This learning method can be used by using as a relationship.

  The functions of some or all of the components in the system configuration diagram shown in FIGS. 1 and 3 can be configured by a computer program, and the program can be executed using a computer to realize the present invention, or It goes without saying that the steps shown in FIGS. 2 and 4 can be configured by a computer program, and the program can be executed by the computer, and the program for realizing the function by the computer, or the processing steps by the computer. The program to be executed is recorded on a computer-readable recording medium, such as a flexible disk, MO, ROM, memory card, CD, DVD, removable disk, etc., and is stored or distributed. Is possible.

  It is also possible to provide the above program through a network such as the Internet or electronic mail. In this way, the present invention can be implemented by installing a program provided by a recording medium or a network in a computer.

1 is a block diagram showing a basic system configuration of a language processing apparatus according to an embodiment of the present invention. The flowchart for demonstrating the fundamental operation | movement of this embodiment. The block diagram which shows the system configuration | structure of the language processing apparatus concerning this embodiment. The flowchart for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the syntax analysis result for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the dependency analysis result among the syntax analysis results for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the update of the syntax analysis result for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the main word table for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the main word table to which the relation clause number is added for explaining the operation of the language processing device Explanatory drawing of the main word table to which the attached word for demonstrating operation | movement of a language processing apparatus was added. Explanatory drawing of the syntax analysis result with respect to the analysis object text for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the dependency analysis result among the syntax analysis results with respect to the analysis object text for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the head table to which the relation clause number and the attached word for demonstrating operation | movement of a language processing apparatus were added. Explanatory drawing of the non-subject predicate table for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the predicate term table with respect to the analysis object text for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the case conversion rule weight (dictionary) with respect to the predicate and action noun of the analysis object text for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the predicate / action noun's use meaning attribute of the analysis object text for explaining operation | movement of a language processing apparatus, and the case conversion rule weight (dictionary) with respect to a predicate / activity noun in general. Explanatory drawing of the term structure table obtained as a result of applying a case conversion rule to the predicate and action noun of the analysis object text for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the term structure table which also added the predicate and action noun which did not have the dependency relationship for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the term structure table after performing the anaphora analysis process for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the term structure table after performing the modifier determination process for demonstrating operation | movement of a language processing apparatus. Explanatory drawing of the term structure table after performing the basic form determination process for demonstrating operation | movement of a language processing apparatus. The block diagram which shows the fundamental system structure of the dictionary creation processing apparatus applied to a language processing apparatus. The flowchart for demonstrating the principle operation | movement of a dictionary creation apparatus. The block diagram which shows the system configuration | structure of a specific dictionary creation apparatus. The flowchart for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the predicate term structure of the correct answer by which the case relation was manually given with respect to the training text for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the term structure correct answer table for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the term structure correct answer table by which the word for demonstrating operation | movement of a dictionary creation apparatus was replaced by the main word. Explanatory drawing which shows the example of the term structure correct answer table to which the dependency relation information for demonstrating operation | movement of a dictionary creation apparatus was added. Explanatory drawing which shows the example of the term structure correct answer table to which the semantic category for demonstrating operation | movement of a dictionary creation apparatus was provided. Explanatory drawing which shows the example of the term structure correct answer table given to the case conversion rule learning part for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the thesaurus for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the case conversion rule produced | generated for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the case conversion rule produced | generated for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the case conversion rule vector for demonstrating operation | movement of a dictionary creation apparatus. Explanatory drawing which shows the example of the case conversion rule and weight obtained by machine learning for demonstrating operation | movement of a dictionary creation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Language processing device 2 ... Case conversion rule table 3 ... Case conversion part 10 ... Language processing device 101 ... Syntax analysis part 102 ... Proper noun connection part 103 ... Subject specification Unit 104 ... dependency dependency analysis unit 106 ... case conversion rule table 110 ... term structure output unit 120 ... syntax analysis result table 121 ... head table 122 ... thesaurus 124 ... Non-subject predicate table 125 ... non-subject predicate storage unit 126 ... case conversion unit 127 ... anaphora analysis unit 128 ... modifier phrase determination unit 129 ... predicate term table 130 ... term structure table

Claims (16)

A language processing device that outputs a term structure about a predicate or a behavioral noun in an input text,
A case conversion rule storage means for storing a rule for converting a dependency state between a predicate or behavioral noun and other words or word attributes into a case relationship between the predicate or behavioral noun and other words;
Applying a rule that converts the dependency state of the text and the case relation of the case conversion rule storage means, and converting the input text into a predicate and a behavioral noun term structure, and outputting the case conversion means,
A language processing apparatus comprising: The rules for converting to the case relationship are:
Generated by machine learning from text with correct case relationship and dependency status,
The language processing apparatus according to claim 1. The dependency state is
About the semantic main part of the phrase, it is the information about the main part and the main part
The language processing apparatus according to claim 1, wherein: The dependency state is
For the semantic headings of clauses, the information of the annexes to the source and destination headings and semantic headings, the part of speech of the semantic head or the semantic category of the semantic head, the predicate or behavioral noun, the semantic attribute or modifier Or information on the genre or category to which the word or text appears in the same text,
The language processing apparatus according to claim 1, wherein: The rules for converting each of the above relationships are:
A parsing unit that parses the text to be analyzed described in a natural language with a term structure attached to a predicate or a behavioral noun, and analyzes a dependency set between clause sets and clauses;
A main part specifying unit that specifies a semantic main part for each phrase from words determined to be independent words in the sentence by the parsing unit, and creates a main part table;
Adding a dependency clause of the clause for each clause of the head table, and adding an auxiliary word that follows in the clause to which the head belongs to the head table dependency analysis unit;
From the term structure of the predicate or behavioral noun assigned to the text, extract the essential surface case for the basic form of the predicate and behavioral noun and the phrase to which each word belongs, create a term structure correct answer table, Each clause of the term structure correct answer table is converted to a main word with reference to the table, and the dependency relation of the clauses in the input text corresponding to the required case of each predicate and action noun is added to the term structure correct answer table. A case conversion rule learning unit for learning a case conversion rule using the term structure correct answer table and a case conversion rule output unit for outputting the obtained predicate and a behavioral noun as a case conversion rule.
The language processing apparatus according to claim 1. A further thesaurus
The case conversion rule learning unit searches the thesaurus, extracts a superordinate concept of words in the term structure correct answer table, further creates a rule that replaces the extracted word and the original word, and The language processing apparatus according to claim 5, wherein the language processing apparatus is added to the structure correct answer table.   The language processing apparatus according to claim 5, wherein the case conversion rule learning unit learns a case conversion rule by machine learning of a correct case conversion rule described in the term structure correct answer table. A case conversion rule storage means for storing a rule for converting a dependency state between a predicate or behavioral noun and other words or word attributes into a case relationship between the predicate or behavioral noun and other words;
Applying a rule for converting the dependency state of the text and the case relationship of the case conversion rule storage means, and converting the input text into a term structure of a predicate and a behavioral noun, and outputting the case conversion means,
A language processing method for outputting a term structure of a predicate or a behavioral noun in an input text,
Entering text and text dependency status;
Applying the case conversion rules stored in the case conversion rule storage means to the predicate or behavioral noun in the text according to the dependency state of the text, and converting to a term structure by the case conversion means;
Outputting a term structure for the predicate or behavioral noun obtained in the above step;
A language processing method comprising: The case conversion rule is:
Generated by machine learning from text with correct case relationship and dependency status,
The language processing method according to claim 8. The dependency state is
About the semantic main part of the phrase, it is the information about the main part and the main part
The language processing method according to claim 8 or 9, wherein The dependency state is
For the semantic headings of clauses, the information of the annexes to the source and destination headings and semantic headings, the part of speech of the semantic head or the semantic category of the semantic head, the predicate or behavioral noun, the semantic attribute or modifier Or information on the genre or category to which the word or text appears in the same text,
The language processing method according to claim 8 or 9, wherein The case conversion rule is:
A step of parsing a text to be analyzed described in a natural language in which a term structure is given to a predicate or a behavioral noun, and analyzing a dependency set between clause sets and clauses;
A main part specifying unit specifying a semantic main part for each phrase from the words determined as independent words in the phrase by the syntactic analysis unit, and creating a main part table;
A dependency relationship analysis unit between the heads adds a dependency clause of the clause for each clause of the head table, and adds, to the head table, an ancillary word that follows in the clause to which the head belongs,
The case conversion rule learning unit extracts, from the term structure of the predicate or behavioral noun assigned to the text, the required surface case for the basic form of the predicate and behavioral noun and the phrase to which each word belongs, and the term structure correct answer A table is created, and each clause of the term structure correct answer table is converted to a head with reference to the head table, and the dependency relationship of clauses in the input text corresponding to the required case of each predicate and action noun is The step of adding to the term structure correct answer table and the case conversion rule output unit are generated from the step of outputting the obtained predicate and the behavioral noun as a case conversion rule.
The language processing method according to claim 8. A further thesaurus
The case conversion rule learning unit searches the thesaurus and extracts a high-level concept of a word in the term structure correct answer table, and then creates a rule that replaces the extracted word with the original word, The language processing method according to claim 12, wherein the language processing method is added to a term structure correct answer table.   13. The language processing method according to claim 12, wherein the case conversion rule learning unit learns a case conversion rule by machine learning of a correct case conversion rule described in the term structure correct answer table. A language processing program for causing a computer to function as each unit in the language processing apparatus according to claim 1.   A computer-readable recording medium on which the language processing program according to claim 15 is recorded.