JP5431532B2 - Question answering apparatus, model learning apparatus, method, and program - Google Patents

Description

  The present invention relates to a question answering device, a model learning device, a method, and a program, and in particular, a question answering device and a model for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence. The present invention relates to a learning apparatus, method, and program.

  A system that outputs answers to natural language questions is called a question answering system. The question answering system is a “factoid type” answering questions that are mainly answered by words, such as “The highest mountain in the world”, “What kind of person is Akira Kurosawa”, and “Why do fireflies shine?” Like this, it is difficult to answer in words, and it is classified as “non-factoid type” that answers in sentences.

  In general, a non-factoid type question answering system answers a question by rearranging answer candidate sentences in an order that seems to be correct. If the correct answer is included in the upper rank, it is a good system. For such rearrangement, a method of combining rules based on empirical rules and similarities between question sentences and answer candidate sentences, and a technique of rearranging based on learning data have been proposed.

  For example, Non-Patent Document 1 relates to a non-factoid type, particularly a method for answering a question asking for a reason. This method first narrows down answer candidate sentences based on whether clue words related to the reason (such as “reason” or “cause”) are included, and then ranks higher in the candidate sentence with many words included in the question sentence. doing. Here, the answer candidate sentences are obtained from newspaper articles.

  Non-Patent Document 2 uses a machine learning method to learn a ranking function that ranks a correct answer higher than an incorrect answer sentence from correct answer and incorrect answer data, and uses that function as a candidate answer sentence. It is about the method of ranking.

  In machine learning, feature quantities are extracted for each pair of a question and an answer candidate sentence, and the weights of the feature quantities are learned from the data. In the above Non-Patent Document 2, as features, coverage of words included in a question sentence, ease of mutual paraphrasing of questions and answer candidate sentences, continuity of question words in answer candidate sentences, questions and answer candidates The degree of sentence co-occurrence is used, and its effectiveness has been reported by experiments using a large amount of Q & A data on the Internet.

  For both Non-Patent Documents 1 and 2 described above, this is a technique based on the assumption that an answer candidate sentence having a high relevance between a question sentence and a semantic content is a good answer.

Shin Morooka, Junichi Fukumoto, “Answer Selection Method for Why-type Question Answer”, IEICE Technical Report Vol.105 No.594 Language Understanding and Communication, IEICE, pp.7-12, 2006. Mihai Surdeanu, Massimiliano Ciaramita, Hugo Zaragoza, “Learning to Rank Answers to Non-Factoid Questions from Web Collections”, Computational Linguistics, vol. 37 (2011), pp. 351-383.

  When answer candidate sentences are extracted from a variety of information sources (for example, the web), answer candidates having different qualities are often obtained because there are various contents. For example, if you are targeting only newspaper articles and Q & A sites, the quality and length of the answer candidate sentences are likely not to change significantly, but on the web there are extremely short and long ones, There are many data that are inappropriate for natural language, such as sentence fragments, pictograms, emoticons, and ASCII art. In order to rank such various answer candidate sentences, if attention is paid only to the relevance of contents, the ranking may be insufficient.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a question answering device, a model learning device, a method, and a program capable of accurately ranking answer candidates for a question sentence.

In order to achieve the above object, a question answering apparatus according to a first invention is a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, Answer candidate extraction means for extracting a set of answer candidates comprising one or more sentences for the inputted question sentence, and each of the answer candidates included in the answer candidate set extracted by the answer candidate extraction means Generating a feature vector that includes a feature amount relating to the relevance between the question sentence and the answer candidate for a combination of the question sentence and the answer candidate, and a feature quantity indicating a ratio of parentheses in the answer candidate And a weight for each feature amount included in the feature vector learned in advance for each combination of the question sentence and the answer candidate, and the feature amount Score calculating means for calculating a score based on the feature vector for the combination generated by the generating means, and ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means. It consists of
A question answering apparatus according to a second aspect of the present invention is a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, wherein 1 for the inputted question sentence Answer candidate extraction means for extracting a set of answer candidates composed of two or more sentences; and for each of the answer candidates included in the answer candidate set extracted by the answer candidate extraction means, the question sentence and the answer candidates For generating a feature vector including a feature amount related to the content relevance between the question sentence and the answer candidate and a feature amount indicating whether or not the first character of the answer candidate is Japanese An amount generating unit, a weight for each feature amount included in the feature vector learned in advance for each combination of the question sentence and the answer candidate, and a feature amount generating unit Score calculating means for calculating a score on the basis of the feature vector for the combination generated in the step, and ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means. It consists of
A question answering apparatus according to a third aspect of the present invention is a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, wherein 1 for the inputted question sentence Answer candidate extraction means for extracting a set of answer candidates composed of two or more sentences; and for each of the answer candidates included in the answer candidate set extracted by the answer candidate extraction means, the question sentence and the answer candidates For generating a feature vector including a feature amount relating to the relevance of the contents of the question sentence and the answer candidate and a feature amount indicating whether or not the last character of the answer candidate is a punctuation point An amount generating unit, a weight for each feature amount included in the feature vector learned in advance for each combination of the question sentence and the answer candidate, and a feature amount generating unit Score calculating means for calculating a score on the basis of the feature vector for the combination generated in the step, and ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means. It consists of

A question answering method according to a fourth invention is a question answering method in a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, comprising answer candidate extraction means To extract a set of answer candidates composed of one or more sentences for the inputted question sentence, and the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by a feature value generation means For each of the above, a feature amount relating to the relevance of the content of the question sentence and the answer candidate with respect to a combination of the question sentence and the answer candidate, and a feature amount indicating a ratio of parentheses in the answer candidate A vector is generated, and each feature included in the feature vector learned in advance for each combination of the question sentence and the answer candidate by the score calculation means. A score is calculated based on the weight for the quantity and the feature vector for the combination generated by the feature quantity generation means, and the set of answer candidates according to the score calculated by the score calculation means by the ranking means Ranking.
A question answering method according to a fifth invention is a question answering method in a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, wherein answer candidate extracting means To extract a set of answer candidates composed of one or more sentences for the inputted question sentence, and the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by a feature value generation means For each combination of the question sentence and the answer candidate, the feature amount relating to the relevance of the contents of the question sentence and the answer candidate, and whether the first character of the answer candidate is Japanese A feature vector including a feature amount, and the feature vector learned in advance for each combination of the question sentence and the answer candidate by a score calculation unit. A score is calculated based on the weight for each feature amount included in the feature vector and the feature vector for the combination generated by the feature amount generation unit, and according to the score calculated by the score calculation unit by the ranking unit, Ranking the set of answer candidates.
A question answering method according to a sixth invention is a question answering method in a question answering apparatus for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence, wherein an answer candidate extracting means To extract a set of answer candidates composed of one or more sentences for the inputted question sentence, and the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by a feature value generation means For each combination of the question sentence and the answer candidate, the feature amount relating to the relevance of the contents of the question sentence and the answer candidate, and whether or not the last character of the answer candidate is a punctuation mark A feature vector including a feature amount, and the feature vector learned in advance for each combination of the question sentence and the answer candidate by a score calculation unit. A score is calculated based on the weight for each feature amount included in the feature vector and the feature vector for the combination generated by the feature amount generation unit, and according to the score calculated by the score calculation unit by the ranking unit, Ranking the set of answer candidates.

A model learning device according to a seventh aspect of the present invention is a model learning device for learning a ranking model for ranking answer candidates of answer sentences for performing non-factoid type question responses to input question sentences. , For each of the answer sentence stored in the question answer data storage means, and a question answer data storage means storing a correct answer sentence consisting of one or more sentences and a non-correct answer sentence for the question sentence, for the combination of the question the answer message, wherein generating a feature amount relating to the relevance of the content of the answer message and the question sentence, the feature vector comprises a feature amount indicating the ratio of the parentheses in the answer message Based on the quantity generation means, the feature vector for each of the answer sentences generated by the feature quantity generation means, and whether or not each of the answer sentences is correct It is configured to include a learning means for learning the weights as the ranking model, for each characteristic amount contained in the feature vector.
A model learning device according to an eighth aspect of the present invention is a model learning device for learning a ranking model for ranking answer candidates of answer sentences for performing non-factoid-type question responses to input question sentences. , For each of the answer sentence stored in the question answer data storage means, and a question answer data storage means storing a correct answer sentence consisting of one or more sentences and a non-correct answer sentence for the question sentence, A feature amount relating to the relevance of the content of the question sentence and the answer sentence with respect to a combination of the question sentence and the answer sentence, and a feature amount indicating whether or not the first character of the answer sentence is Japanese Feature quantity generation means for generating a feature vector; whether the feature vector for each of the answer sentences generated by the feature quantity generation means and each of the answer sentences is correct Based on the transient, and is configured to include a learning means for learning the weights as the ranking model, for each characteristic amount contained in the feature vector.
A model learning device according to a ninth aspect of the present invention is a model learning device that learns a ranking model for ranking answer candidates of answer sentences for performing non-factoid type question responses to input question sentences. , For each of the answer sentence stored in the question answer data storage means, and a question answer data storage means storing a correct answer sentence consisting of one or more sentences and a non-correct answer sentence for the question sentence, A feature amount related to the relationship between the question sentence and the answer sentence with respect to a combination of the question sentence and the answer sentence, and a feature amount indicating whether or not the last character of the answer sentence is a punctuation mark Feature quantity generation means for generating a feature vector; whether the feature vector for each of the answer sentences generated by the feature quantity generation means and each of the answer sentences is correct Based on the transient, and is configured to include a learning means for learning the weights as the ranking model, for each characteristic amount contained in the feature vector.
A program according to a tenth invention is a program for causing a computer to function as each means of the above question answering device or the above model learning device.

  As described above, according to the question answering device, model learning device, method, and program of the present invention, for each set of answer candidates for the question sentence, the sentence length, character type, sentence validity in the answer candidate, and By generating a feature vector that includes a feature quantity related to at least one of the presence or absence of an interrogative word, calculating a score, and ranking a set of answer candidates according to the score, the answer candidates for the question sentence are ranked with high accuracy. The effect that it can be obtained.

It is the schematic which shows the structure of the model learning apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a morphological analysis result. It is a figure which shows an example of the feature value of each feature-value. It is a figure which shows an example of the feature name and feature value of each feature-value. It is the schematic which shows the structure of the question answering apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the model learning process routine in the model learning apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the question answer generation process routine in the question answering apparatus which concerns on embodiment of this invention. It is a figure which shows an experimental result.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline of the invention>
In order to accurately rank miscellaneous answer candidate sentences, attention is paid to features other than the contents. More specifically, the ranking function is learned by a machine learning technique using feature quantities relating to sentence length, character type, sentence validity, and presence / absence of an interrogative word. These features eliminate sentences that are too long or too short, exclude sentences that are not natural, such as web page fragments, prefer sentences that end correctly, and ask questions. This is to avoid answering questions. For example, since there are a large number of question sentences on the web, the feature quantities related to the presence or absence of question words are exactly the same as the user's question sentences when ranking sentences based only on the contents of conventional questions In some cases, similar questions may be ranked in the top as answers, and this feature is introduced to avoid it.

<System configuration>
Hereinafter, a case where the present invention is applied to a non-factoid type, in particular, a question answering system that answers a reason will be described as an example.

  Here, the question answering system that answers the reason searches the web or text database in response to the input of the question sentence that asks the reason, extracts a set of sentences from the resulting set of documents, and extracts these appropriately. Answers shall be made by ranking in order. In this system, for the purpose of returning a response as short and concise as possible, a single sentence is extracted as a unit. However, if you want to add more content to the answer, answer multiple phrases or paragraphs. It is good. In that case, the unit of learning data, which will be described later, also needs to be matched with this unit.

  As shown in FIG. 1, a model learning apparatus 100 of a question answering system according to an embodiment of the present invention receives correct answer data and learns and outputs a ranking model. This model learning apparatus 100 is composed of a computer including a CPU, a RAM, and a ROM that stores a program for executing a model learning processing routine described later, and is functionally configured as follows. Yes. As shown in FIG. 1, the model learning device 100 includes an input unit 10, a calculation unit 20, and an output unit 30.

  The input unit 10 receives a question sentence, one or more correct answer sentences as answers to the question sentence, and one or more incorrect answer sentences as a set, and data including one or more such sets. Accepted as correct answer data. For example, for each of a plurality of question sentences, a data set including the question sentence, a plurality of correct answer sentences that are answer sentences to the question sentence, and a plurality of incorrect answer sentences that are not answer sentences to the question sentence is received.

  In this embodiment, 150 question sentences are prepared as correct answering data. For example, the following are some of them:

・ Why is it said that the brain is a scary disease?
・ Why is drinking too much wine bad for your body?
・ While there is a “Faust” in the foreign translation, the reason why another translation is published ・ Why was the Toei Rengo movie boom in the 1960s?

  And from each question sentence, only the content word was taken out and the search query was produced | generated. For example, if “why is it said that the brain is a scary disease?”, Morphological analysis is first performed, and an output (morphological analysis result) as shown in FIG. 2 is obtained. Here, JTAG developed by the applicant is used as the morphological analysis engine.

  In FIG. 2, one line corresponds to one word, and in each line, word (surface), part of speech, standard form, final form (if any), and semantic attribute information based on the Japanese vocabulary system are output. . The semantic attribute information based on the Japanese vocabulary system is output for each word based on the word dictionary included in the morphological analysis engine.

  If there is a word that matches the following list of interrogative words in the standard form field from the set of words obtained from the morphological analysis result, it is first removed.

Why, who, where, when, how, how, how, how, how, how, how, what, what, what, what, what, what, what

  Here, “why” matches, so this line is deleted first. Subsequently, from the line corresponding to the noun, the verb stem, and the adjective stem, the standard form (the final form if there is a final form) is extracted, and these are used as search words. For example, in the case of the above example, the search word is “brainstick, illness, scary”.

  These words are separated by blanks, search queries are generated, web searches are performed, and the top 100 search results are obtained. Here, a web search engine that is generally widely used is used. These search results are given a summary text called a snippet in addition to the URL. These are excerpts of the URL content, which consists of about 100 to 150 characters. In the present embodiment, the snippet is divided by a delimiter such as a punctuation mark or “...” And divided into sentences. Several sentences are obtained from each search result, and several hundred sentences are obtained from 100 search results.

  For example, the following is a part of a set of sentences extracted based on a search result obtained by using a query “brain disease is scared”. It is one line one sentence. A total of 421 sentences were obtained from the search results.

Many cerebral infarctions are very scary when older people get sick.
Many people lose their lives due to this disease called cerebral infarction, and many people are fighting the aftereffects of paralysis like their daughters.
This site was opened because we wanted to make as many people as possible aware of the sequelae of hemiplegia and to be a place for interaction with those suffering from sequelae.
Please see, September 5, 2008 That is a scary illness called cerebral infarction or myocardial infarction.
If blood vessels swell, if the blood pressure is too high, it will swell like a balloon and eventually rupture.
This is called an aneurysm or varicose vein.

  About these sentences, the correct sentence was tagged manually. Specifically, the above sentence was read manually, it was determined whether the answer was a question, and the correct sentence was selected. Sentences not selected here are incorrect sentences. Here, 22 sentences including the following sentences were selected as correct sentences from 421 sentences. The first number is the line number in 421 sentences.

1 Many cerebral infarctions are very scary if older people get sick.
2 Many people lose their lives due to the disease called cerebral infarction, and many people are fighting the aftereffects of paralysis like their daughters.
36 The cerebral infarction disease is a scary disease that suddenly attacks any place.
49 A cerebral infarction is a very scary illness that kills your body in an instant, and in the worst case is fatal.
60 Stroke is a very scary illness, because even if life is saved by emergency treatment, paralysis is often left behind or bedridden.

  In the present embodiment, data including such correct answer sentences and incorrect answer sentences is created for all 150 questions and used as correct answer question response data.

  The calculation unit 20 includes a correct answer / answer data storage unit 21, a feature amount generation unit 22, and a model learning unit 23.

  The correct answer / answer data storage unit 21 stores correct answer / answer data received by the input unit 10.

  The feature quantity generation unit 22 generates a feature vector for each combination of the question sentence Q obtained from the correct answer response data and each sentence C in the sentence set including the correct answer sentence and the incorrect answer sentence.

  Here, in the present embodiment, “reciprocal number of sentence order (rank)”, “number of words (len)”, “semantic attribute about reason (ncat-semantic attribute)”, “whether question is included (question) ”,“ First character is Japanese (sentence-begin) ”,“ Last character is a punctuation point (sentence-end) ”,“ Japanese percentage (ja) ”,“ Space and punctuation percentage (common) ” , “Latin ratio (latin)”, “Parent ratio (paren)”, “Symbol ratio (symbol)”, “Content word coverage (cw-count)”, “Word (surface layer) coverage (count) ) ”,“ Word (standard form) coverage (base-count) ”,“ semantic attribute coverage (cats-count) ”, and“ includes cause expression pattern (synpat-pattern) ” Generate feature vectors as elements. For the question sentence Q and the sentence C, it is assumed that morpheme analysis is performed, and the same morpheme analysis result as in FIG. 2 is obtained. The name in parentheses is the feature name. Note that “sentence-begin” and “sentence-end” are examples of features related to the validity of the answer candidate sentence. "Ratio (ja)", "Space and punctuation ratio (common)", "English character ratio (latin)", "Parent ratio (paren)", and "Symbol ratio (symbol)" It is an example of the feature-value regarding. In addition, “reciprocal number of sentence rank (rank)”, “content word coverage (cw-count)”, “word (surface layer) coverage (count)”, “word (standard form) coverage (base-count) ”And“ Semantic attribute coverage (cats-count) ”are examples of feature amounts relating to the relevance of the contents of question sentences and answer candidates. Among the above feature quantities, “number of words (len)” and “whether it contains a question word” to “ratio of symbols” are examples of the feature quantities proposed in the present invention. Others are known in the art, and their effectiveness has been confirmed.

  “Reciprocal number of sentence order (rank)” is the reciprocal number of the relative order of sentence C when the sentence set obtained for the question sentence Q is arranged according to the search order in the snippet. For example, if there are 421 sentences and C is the first sentence, it becomes 1/421.

  “Number of words (len)” is the number of words included in the sentence C, and is an example of a sentence length.

  Referring to the Japanese vocabulary system, there are seven semantic attributes of words related to “reason”: 2448 and 2450-2456. Therefore, “semantic attribute about reason (ncat-semantic attribute)” is seven binary feature amounts indicating whether or not words of seven semantic attributes are included in the sentence C, and only for the semantic attributes included in C, Set the value to 1. Here, since the question about the reason is dealt with, the presence of these semantic attributes is used as the feature quantity. However, when answering “method”, the semantic attribute corresponding to the method may be used.

  In “question included” (question), it is 1 if the word included in the above list of question words is in sentence C.

  “Sentence-begin” is “1” if the first character of sentence C is a Japanese character (Kanji, Hiragana, Katakana).

  “Sentence-end” is “1” if the last character of sentence C is a punctuation mark.

  “Proportion of Japanese (ja)” is the ratio of Japanese characters (Kanji, Hiragana, Katakana) in the characters of sentence C. For example, if 5 out of 10 characters are kanji, hiragana, or katakana, then 0.5.

  “The ratio of space and punctuation (common)” is the ratio of space and punctuation in the characters of sentence C.

  The “letter ratio (latin)” is the ratio of the letters of the letter C in the letters of the sentence C.

  “Ratio of other characters (other)” is the proportion of characters in sentence C that do not match any of Japanese, space, punctuation, or alphabetic characters.

  The “paren” is the ratio of words in sentence C whose part of speech is parentheses.

  The “symbol ratio” is the ratio of words in the sentence C whose part of speech is Symbol. The word whose part of speech is Symbol is, for example, a colon or a period.

  “Content word coverage (cw-count)” is the number (cwcount) of content words included in the sentence C among the content words included in the question sentence Q. Further, the number of content words (cw-count-rev) included in the question sentence Q among the content words included in the sentence C may be used as the feature amount. Further, the ratio (cw-cov) of content words included in the sentence C among the content words included in the question sentence Q may be used as the feature amount. Further, the ratio (cw-cov-rev) of the content words included in the question sentence Q among the content words included in the sentence C may be used as the feature amount. Here, the content words are those whose part of speech is a noun, a verb stem, an adjective stem, and an unknown word.

  “Word (surface layer) coverage (count)” is the number (count) of words (surface layer) included in the sentence C among the words (surface layer) included in the question sentence Q. Further, as the feature amount, the number (count-rev) of words included in the question sentence Q among the words (surface layer system) included in the sentence C may be used. In addition, as a feature amount, a ratio (cov) of words included in the sentence C among the words included in the question sentence Q (surface layer system) (cov) may be used. The ratio (cov-rev) included in the question sentence Q may be used. Similarly, in addition to words (unigrams), for bigrams (word continuation), the number and ratio (bi-count, bi-cov, bi-count-rev, bi-cov-rev) are similarly calculated. It may be used as a feature amount.

  “Word (standard form) coverage (base-count)” is a feature quantity similar to the coverage of a word (surface layer system), and is a value for the standard form of the word. Further, a value (base-cov) relating to the ratio may be used.

  “Semantic attribute coverage (cats-count)” is a feature quantity similar to content word coverage, and is the number of noun semantic attributes included, not the content word. Further, a value (cats-cov) regarding the ratio may be used.

  `` Do you include a cause expression pattern (synpat-pattern) '' means that there is a set of patterns representing causes that are prepared in advance, and whether or not sentence C matches these patterns and the binary feature amount for each pattern It is what. These patterns can be created by removing other than function words (particles, auxiliary verbs) from phrases or the like that are known in advance as the reason is written. For example, in the present embodiment, “it is *”, “* is *”, “ga * is”, “is * is”, “is * is *”, “is * is *” Was used as a pattern. An asterisk is a symbol that matches anything other than a function word. Whether sentence C matches each pattern is a feature amount. Since a total of 490 patterns are used, 490 binary feature values are generated. How to make a pattern here is written in Non-Patent Document 3 (Ryuichiro Higashinaka, Hideki Isozaki, “Corpus-based Question Answering for why-Questions”, IJCNLP, vol.1, pp.418-425, 2008.). It is the same as the method.

  Next, an example of feature vector generation is shown. Each feature obtained from Q “Why is it said that the brain is a scary disease?” And C “Many cerebral infarctions are afraid even if the elderly develops the disease.” The first +1 indicates that the sentence C is a correct sentence, and qid is a question ID assigned to the question sentence Q. Thereafter, the feature quantity (feature) number, colon, and feature value are separated by a space.

  When this feature quantity is tabulated together with the feature name, it is as shown in FIG.

  The model learning unit 23 learns a ranking model by machine learning based on the feature amount obtained for each pair of the question sentence Q and the sentence C. For a certain question sentence Q, sentence C is a correct answer sentence, sentence C 'is an incorrect answer sentence, and feature quantity vectors obtained from question sentence Q and sentence C are obtained from question sentence Q and sentence C'. Let the feature vector be F ′. Here, if there is a certain weight vector W, each score (obtained by the inner product) is compared, and the correct sentence is found by finding a W that makes FW larger than F'W by an optimization method or the like. A weight vector (ranking model) that ranks higher than the incorrect answer sentence is obtained. Such a weight vector can be obtained by using a ranking SVM, which is a method generally used for ranking learning. In this embodiment, a feature vector is generated from 150 questions and the correct and incorrect sentences obtained for each question, and for each question, the correct sentence is ranked higher than the incorrect sentence. W is obtained by ranking SVM. The ranking SVM used a linear kernel.

  The ranking model learned by the model learning unit 23 is output by the output unit 30.

  As shown in FIG. 5, the question answering apparatus 150 of the question answering system according to the embodiment of the present invention receives a question sentence, ranks and outputs answer candidate sentences for the question sentence. This question answering apparatus 150 is composed of a computer including a CPU, a RAM, and a ROM storing a program for executing a question answer generation processing routine described later, and is functionally constructed as follows. ing. As shown in FIG. 5, the question answering apparatus 150 includes an input unit 60, a calculation unit 70, and an output unit 80.

  The input unit 60 receives the input question sentence.

  The calculation unit 70 includes a morpheme analysis unit 71, a search query generation unit 72, a search unit 73, a sentence set extraction unit 74, a morpheme analysis unit 75, a feature quantity generation unit 76, a ranking unit 77, and a model storage unit 78. . The sentence set extraction unit 74 is an example of answer candidate extraction means. The ranking unit 77 is an example of a score calculation unit and a ranking unit.

  The morpheme analysis unit 71 obtains a morpheme analysis result similar to that in FIG. 2 by performing morpheme analysis on the question sentence received by the input unit 10.

  The search query generation unit 72 generates a search query using words included in the question sentence based on the morphological analysis result of the question sentence, as in the case of learning the ranking model.

  Using the generated search query, the search unit 73 performs a search process on the Web via the Internet using a web search engine, and obtains a search result.

  The sentence set extraction unit 74 extracts each snippet included in the search result, divides each snippet into sentences, and uses the obtained sentence set as the sentence of the answer candidate sentence, as in the case of learning the ranking model. Let it be a set.

  The morpheme analysis unit 75 performs morpheme analysis on each of the sentence sets of the answer candidate sentences, and obtains the same morpheme analysis result as in FIG.

  For each of the sentence sets of the answer candidate sentences, the feature quantity generation unit 76 performs the above-described feature quantity generation unit based on the respective morphological analysis results for the combination of the question sentence received by the input unit 10 and the answer candidate sentence. Similarly to “22,“ reciprocal number of sentence order (rank) ”,“ number of words (len) ”,“ semantic attribute about reason (ncat-semantic attribute) ”,“ whether it contains a question (question) ”,“ one character `` Eyes are Japanese (sentence-begin) '', `` Last letter is a punctuation mark (sentence-end) '', `` Japanese ratio (ja) '', `` Space and punctuation ratio (common) '', `` English characters "Rate (latin)", "parentheses ratio (paren)", "symbol ratio (symbol)", "content word coverage (cw-count)", "word (surface layer) coverage (count)", " Word (standard form) coverage (base-count) "," semantic attribute coverage (cats-count) ", and" cause expression pattern included (synpat-pattern Generating a feature vector for each feature quantity including a down) "and elements.

  The model storage unit 78 stores the ranking model (weight vector for each feature amount) output by the model learning device 100.

  For each combination of the question sentence and the answer candidate sentence received by the input unit 10, the ranking unit 77 stores a weight vector (ranking model) stored in the model storage unit 78 and a feature vector generated for the combination. By multiplying and, the score is calculated, and the answer candidate sentences are ranked according to the calculated score. All of these answer candidate sentences or the top N cases are output by the output unit 80 to be system answers.

<Operation of question answering system>
Next, the operation of the question answering system according to the present embodiment will be described. First, when correct answer question response data prepared in advance is input to the model learning device 100, the input correct question answer data is stored in the correct question answer data storage unit 21 by the model learning device 100. Then, the model learning processing routine shown in FIG.

  First, in step S101, all the question sentences, correct sentences, and incorrect sentences included in the correct answer data are acquired from the correct answer data storage unit 21. In step S102, a combination of a question sentence and a correct answer sentence or an incorrect answer sentence is selected from the data acquired in step S101.

  In step S103, for the combination of the question sentence selected in step S102 and the correct answer sentence or incorrect answer sentence, based on the morphological analysis result of the question sentence and the morphological analysis result of the correct sentence or incorrect answer sentence, Generate a feature vector.

  In the next step S104, it is determined whether or not the processes in steps S102 and S103 have been executed for all combinations of correct answer / answer data. If there is a combination for which the processes in steps S102 and S103 are not executed, the process returns to step S102, and the combination is selected. On the other hand, if it is determined that the processing of steps S102 and S103 has been executed for all combinations of correct question response data, the process proceeds to step S105.

  In step S105, based on the feature vectors of all combinations generated in step S103, the weight vector (ranking) that is the weight of each feature quantity of the feature vector is ranked higher than the incorrect answer sentence. Model) by machine learning.

  In step S106, the ranking model learned in step S105 is output by the output unit 30, and the model learning processing routine is terminated.

  Then, the ranking model output by the output unit 30 is stored in the model storage unit 78 of the question answering device 150. When the question text is input to the question answering device 150, the question answering device 150 executes a question answer generating process routine shown in FIG.

  In step S111, the inputted question sentence is received. In step S112, morphological analysis is performed on the question sentence received in step S111.

  In step S113, a search query is generated based on the morphological analysis result of the question sentence obtained in step S112. In step S114, search processing is performed on the Web using the search query generated in step S113.

  In the next step S115, a set of answer candidate sentences is extracted from the search result obtained in step S114. In step S116, morphological analysis is performed on each answer candidate sentence included in the set of answer candidate sentences extracted in step S115.

  In step S117, a feature vector is generated for each combination of the inputted question sentence and each answer candidate sentence. In step S118, using the ranking model and the feature vector generated in step S117, a score is calculated for each combination of the input question sentence and each answer candidate sentence.

  In the next step S119, according to the score calculated in step S118, the set of answer candidate sentences is ranked and output by the output unit 80, and the response generation processing routine is terminated.

<Example>
Examples are shown below. 150 questions were divided into 5 and cross-validated. Top-N Mean Reciprocal Rank (MRR) and N-rank accuracy rate were used as the evaluation scale. Top-N MRR is the average of the scores for a given set of questions, when the top N answers are output for a question, the reciprocal of the ranking of the top answer is the score for that question. Is what we asked for. The N-rank correct answer rate is the percentage of questions that included correct answers when looking at the output up to the N-th rank. For example, if there are 10 questions and the system output includes 5 answers up to 5th place, 5/10 = 0.5 is the 5th place correct answer rate. Features proposed in the present invention (“question” (question), “first character is Japanese (sentence-begin)”, “last character is punctuation (sentence-end)”, “Japanese ) (Character ratio (ja) "," space and punctuation ratio (common) "," letter ratio (latin) "," parentheses ratio (paren), "symbol ratio (symbol)") The case where a feature vector as an element was used was taken as a comparative example. The result of cross-validation is shown in FIG.

  Features proposed in the present invention (“question” (question), “first character is Japanese (sentence-begin)”, “last character is punctuation (sentence-end)”, “Japanese "Feature ratio (ja)", "space and punctuation ratio (common)", "letter ratio (latin)", "parentheses ratio (paren)", "symbol ratio (symbol)") As a result, improvement was observed in both the top-N MRR and the N-rank accuracy rate, and the effectiveness of the feature amount proposed in the present invention could be confirmed.

  As described above, according to the question answering apparatus according to the present embodiment, for each set of answer candidate sentences for the question sentence, the sentence length, the character type, the validity of the sentence, and the existence of the question word in the answer candidate sentence By generating a feature vector including a feature quantity relating to each of the presence or absence of a sentence, calculating a score, and ranking a set of answer candidate sentences according to the score, the answer candidates for the question sentence can be accurately ranked.

  In addition, since the answer candidate sentences are appropriately ranked with respect to the question, it is possible to realize a system that accurately outputs the answer to the user's question, thereby improving the efficiency of the user's intellectual activities.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, the model learning device and the question answering device may be realized by a single device. In this case, the ranking model learned by the model learning unit may be stored in the model storage unit.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

21 correct answer / answer data storage unit 22 feature quantity generation unit 23 model learning unit 60 input unit 70 calculation unit 71 morpheme analysis unit 72 search query generation unit 73 search unit 74 sentence set extraction unit 75 morpheme analysis unit 76 feature quantity generation unit 77 ranking Unit 78 Model storage unit 100 Model learning device 150 Question answering device

Claims (14)

A question answering device for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence,
Answer candidate extraction means for extracting a set of answer candidates comprising one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means, the relevance of the contents of the question sentence and the answer candidate with respect to a combination of the question sentence and the answer candidate Feature quantity generating means for generating a feature vector including a feature quantity and a feature quantity indicating a ratio of parentheses in the answer candidates;
For each combination of the question sentence and the answer candidate, based on a weight for each feature amount included in the feature vector learned in advance and the feature vector for the combination generated by the feature amount generation means , A score calculating means for calculating a score;
Ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means;
Including question answering device. A question answering device for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence,
Answer candidate extraction means for extracting a set of answer candidates comprising one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means, the relevance of the contents of the question sentence and the answer candidate with respect to a combination of the question sentence and the answer candidate Feature quantity generating means for generating a feature vector including a feature quantity and a feature quantity indicating whether or not the first character of the answer candidate is Japanese ;
For each combination of the question sentence and the answer candidate, based on a weight for each feature amount included in the feature vector learned in advance and the feature vector for the combination generated by the feature amount generation means , A score calculating means for calculating a score;
Ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means;
Including question answering device. A question answering device for generating an answer sentence for performing a non-factoid type question answer to an inputted question sentence,
Answer candidate extraction means for extracting a set of answer candidates comprising one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means, the relevance of the contents of the question sentence and the answer candidate with respect to a combination of the question sentence and the answer candidate Feature quantity generating means for generating a feature vector including a feature quantity and a feature quantity indicating whether or not the last character of the answer candidate is a punctuation point ;
For each combination of the question sentence and the answer candidate, based on a weight for each feature amount included in the feature vector learned in advance and the feature vector for the combination generated by the feature amount generation means , A score calculating means for calculating a score;
Ranking means for ranking the set of answer candidates according to the score calculated by the score calculating means;
Including question answering device.   The feature amount generation means includes, for each of the answer candidates included in the answer candidate set extracted by the answer candidate extraction means, a sentence length, presence / absence of a question word, and presence of a semantic attribute in the answer candidate The question answering device according to any one of claims 1 to 3, wherein the feature vector further including a feature quantity related to at least one of presence or absence of is generated. Search query generation means for generating a search query based on the inputted question sentence;
Search means for performing a search process on the Web using the search query generated by the search query generation means,
The answer candidate extraction means extracts the set of answer candidates from the search results obtained by the search processing by the search means,
The feature quantity generating unit generates each feature vector of the combination based on a morphological analysis result of the question sentence and a morphological analysis result of each of the answer candidates included in the set of answer candidates. The question answering device according to any one of claims 1 to 4 . A model learning device for learning a ranking model for ranking answer candidates of answer sentences for performing non-factoid type question responses to input question sentences,
Question answer data storage means for storing a correct answer sentence composed of one or more sentences and a non-correct answer sentence for the question sentence;
For each of the answer sentences stored in the question answer data storage means, a feature amount relating to the relevance of the contents of the question sentence and the answer sentence with respect to a combination of the question sentence and the answer sentence, and the answer sentence Feature quantity generating means for generating a feature vector including a feature quantity indicating a ratio of parentheses in the middle ;
Based on the feature vector for each of the answer sentences generated by the feature quantity generation means and whether or not each of the answer sentences is correct, a weight is assigned to each feature quantity included in the feature vector. Learning means to learn as a model,
Model learning device including A model learning device for learning a ranking model for ranking answer candidates of answer sentences for performing non-factoid type question responses to input question sentences,
Question answer data storage means for storing a correct answer sentence composed of one or more sentences and a non-correct answer sentence for the question sentence;
For each of the answer sentences stored in the question answer data storage means, a feature amount relating to the relevance of the contents of the question sentence and the answer sentence with respect to a combination of the question sentence and the answer sentence, and the answer sentence Feature quantity generating means for generating a feature vector including a feature quantity indicating whether or not the first character of the character is in Japanese ,
Based on the feature vector for each of the answer sentences generated by the feature quantity generation means and whether or not each of the answer sentences is correct, a weight is assigned to each feature quantity included in the feature vector. Learning means to learn as a model,
Model learning device including A model learning device for learning a ranking model for ranking answer candidates of answer sentences for performing non-factoid type question responses to input question sentences,
Question answer data storage means for storing a correct answer sentence composed of one or more sentences and a non-correct answer sentence for the question sentence;
For each of the answer sentences stored in the question answer data storage means, a feature amount relating to the relevance of the contents of the question sentence and the answer sentence with respect to a combination of the question sentence and the answer sentence, and the answer sentence Feature quantity generating means for generating a feature vector including a feature quantity indicating whether or not the last character of
Based on the feature vector for each of the answer sentences generated by the feature quantity generation means and whether or not each of the answer sentences is correct, a weight is assigned to each feature quantity included in the feature vector. Learning means to learn as a model,
Model learning device including The feature amount generating means relates to at least one of the sentence length, the presence / absence of a question word, and the presence / absence of a semantic attribute in each answer sentence stored in the question / answer data storage means. The model learning device according to claim 6, wherein the feature vector further including a feature amount is generated. A question answering method in a question answering device for generating an answer sentence for performing a non-factoid type question answering to an inputted question sentence,
A candidate answer extracting unit extracts a set of answer candidates including one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by the feature quantity generation means, the question sentence and the answer candidate for the combination of the question sentence and the answer candidate Generating a feature vector including a feature amount relating to the relevance of the contents of the response and a feature amount indicating a ratio of parentheses in the answer candidate;
For each combination of the question sentence and the answer candidate by the score calculation means, a weight for each feature quantity included in the feature vector learned in advance and the feature for the combination generated by the feature quantity generation means Calculate the score based on the vector,
A question answering method in which ranking means ranks the set of answer candidates according to the score calculated by the score calculation means . A question answering method in a question answering device for generating an answer sentence for performing a non-factoid type question answering to an inputted question sentence,
A candidate answer extracting unit extracts a set of answer candidates including one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by the feature quantity generation means, the question sentence and the answer candidate for the combination of the question sentence and the answer candidate Generating a feature vector that includes a feature amount related to the relevance of the content and a feature amount indicating whether or not the first character of the answer candidate is Japanese ,
For each combination of the question sentence and the answer candidate by the score calculation means, a weight for each feature quantity included in the feature vector learned in advance and the feature for the combination generated by the feature quantity generation means Calculate the score based on the vector,
A question answering method in which ranking means ranks the set of answer candidates according to the score calculated by the score calculation means . A question answering method in a question answering device for generating an answer sentence for performing a non-factoid type question answering to an inputted question sentence,
A candidate answer extracting unit extracts a set of answer candidates including one or more sentences for the inputted question sentence;
For each of the answer candidates included in the set of answer candidates extracted by the answer candidate extraction means by the feature quantity generation means, the question sentence and the answer candidate for the combination of the question sentence and the answer candidate Generating a feature vector that includes a feature amount relating to the relevance of the content of and a feature amount indicating whether or not the last character of the answer candidate is a punctuation point
For each combination of the question sentence and the answer candidate by the score calculation means, a weight for each feature quantity included in the feature vector learned in advance and the feature for the combination generated by the feature quantity generation means Calculate the score based on the vector,
A question answering method in which ranking means ranks the set of answer candidates according to the score calculated by the score calculation means . The computer program to function as the respective means constituting the question answering equipment of any one of claims 1 to 5. Program for causing a computer to function as each unit constituting the model learning device according to any one of claims 6 to claim 9.