JP2010244498A - Automatic answer sentence generation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem in conventional automatic answer sentence generation in which since it is uncertain for what kind of items a user has a dialogue on a posting board, blog or chat on the Internet, and the topic frequently changes in accordance with the dialogue, it is impossible to prepare an answer sentence or knowledge for an item to be answered, and an answer sentence having no matching property can be generated. <P>SOLUTION: A key word is extracted from language information transmitted by a person, information on Webs in the Internet is collected for this keyword to complement the keyword, and such a keyword is verified with a field association word dictionary 5, thereby determining types of topics of the language information transmitted by the person. Thereafter, sentence generation is performed based on the keyword extracted from the language information transmitted by the person, the field information thereof, and a text template that is a model of answer sentence, thereby generating an answer sentence appropriate to a topic, even when it is an unspecified topic. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a system that understands a topic for a conversation sentence on an unspecified topic such as a bulletin board, a blog, or a chat on the Internet and generates an appropriate response sentence for the topic.

As conventional automatic response sentence generation, there is an operation method guidance guide that guides what operation should be performed for a specific situation on a specific device or the screen on the Internet. In addition, there is a means to search similar information in a manual or Q & A collection prepared in advance for a question sentence on the same specific device or Internet screen, and display the search result as a response sentence. Do it.
In addition, there are various efforts in academic research on automatic response sentence generation, and many researchers are working on this issue. As a report of the research results, documents shown in [Non-patent document 1], [Non-patent document 2] and [Non-patent document 3] have been reported.
Yoji Kiyota, Ikuo Kurohashi, Toko Kido: “Automatic Question Answering Based on Large-Scale Text Knowledge Base-Dialog Navi”, Journal of Natural Language Processing, Vol. 10, no. 4, pp. 145-175, 2003. Atsushi Matsumoto, Kiyoaki Shirai: “Question answering system that detects multiple ambiguities and presents multiple answers”, Proc. 935-938, 2006. Atsushi Sakamoto: “Study on the generation of question answering in interactive question answering system”, Master'ssis, Japan Advanced Institute of Science and Technology, 2007. Hiroshi Nakagawa, Akinori Mori, Akira Yumoto: “Extraction of technical terms based on appearance frequency and connection frequency”, Journal of Natural Language Processing, Vol. 10, no. 1, pp. 27-45, 2003 Junji Umemura: “Development of keyword extraction system for unexplored text information”, Unexplored Software Creation Project, 2000

  As conventional automatic response sentence generation, as shown in the above [0002], there is a technique for generating a response sentence for a specific situation on a specific device or a screen on the Internet, but an unspecified situation. There is no response sentence generation system for.

  This is because if you are targeting a specific situation on a specific device or screen on the Internet, you can limit the situation, so what the user will do next in that situation, what to think about in that situation, etc. It is possible to limit the items that should be answered in advance. For this reason, it is possible to prepare in advance the knowledge of the response text or the matters to be answered, but it is not clear what the user will interact with on the Internet bulletin board, blog, chat, etc. Since the topic often changes as the progress of, it is impossible to prepare the knowledge of the response sentence or the item to be responded in advance.

  In addition, knowledge such as existing national dictionaries, synonym dictionaries, a thesaurus that expresses the semantic hierarchical structure between words, and words needed to identify topics such as nouns and proper nouns from documents are extracted as keywords. There is already a keyword extraction technique from the sentence to be used, but there is a technique to identify the topic area by comparing the keywords required to identify the topic extracted by the keyword extraction technique with a thesaurus such as thesaurus. . However, on bulletin boards, blogs, and chats on the Internet, abbreviations that combine multiple words into one, abbreviations that combine English initials, and common names are often used to describe documents. In many cases, it is not possible to collate words extracted from documents with keyword extraction technology with existing Japanese dictionaries, synonym dictionaries, or thesaurus that expresses the semantic hierarchical structure between words. In some cases, the accuracy rate in the identified field is low. Therefore, there is a problem that even if a response sentence corresponding to the specified field is generated, the response sentence is completely inconsistent.

  In addition, because there are many homonyms in the words that identify the field, it is possible to accurately extract words that are sufficient to identify the field using the keyword extraction technology, and the extracted words and existing Japanese dictionary or Even if collation with a thesaurus or a thesaurus that expresses a semantic hierarchical structure between words is executed correctly, the identified field is wrong, and even if a response sentence corresponding to the identified field is generated, The problem arises that the response sentence is completely inconsistent.

  In order to solve such a problem, in the present invention, a system for inputting a conversation sentence on an unspecified topic such as a bulletin board, a blog, a chat on the Internet on the Internet, a guidance system for various device operation and usage methods, and various exhibitions Language information input means for capturing language information sent by humans in an environment where the machine or system responds to language information sent by human writing or speaking, such as a guidance system in a venue or company, and language information Keyword extracting means for extracting a keyword as information for determining a field from the keyword, Web keyword complementing means for collecting information on the Internet on the Internet for language information transmitted by humans and complementing the keyword, and keyword and field Using a field-associative dictionary that stores name correspondence information, Field information acquisition means for acquiring the field information to which each keyword belongs by collating the keyword extracted by the keyword extraction means with the keyword supplemented by the Web keyword complementing means and the field associative word dictionary with respect to the language information transmitted between them; A field hierarchical structure dictionary storing a hierarchical structure between fields, a field extracted by a keyword extracting means for language information transmitted by a person, field information to which a keyword complemented by a web keyword complementing means belongs, and a field hierarchical structure dictionary Field inference means that infers and outputs fields by collation, response sentence dictionary that stores sentence templates that serve as response sentence templates, keywords extracted by language extraction means for language information sent by humans, and field inference The topic inferred by means is connected to a sentence template that serves as a template for the reply sentence stored in the reply sentence dictionary. An automatic response sentence generation system that generates an appropriate response sentence for a topic even if it is an unspecified topic with respect to linguistic information transmitted by a human adopt.

  Furthermore, it corresponds to abbreviations that combine multiple words into one, abbreviations that combine English initials, and words that are not registered in existing dictionaries such as national dictionaries but are commonly used. And means for increasing the number of words serving as related information, and Web search means for searching a Web page on the Internet using a keyword extracted by a keyword extraction means for language information transmitted by a human. , Keyword information is extracted from human language information by using Web keyword aggregating means for extracting keywords by keyword extracting means and aggregating the extracted keywords with respect to the linguistic information as a result of searching by Web searching means. Adopt Web keyword complementing means characterized by complementing.

  Furthermore, since the field to which this set belongs is inferred from the set of keywords and the field information to which the keyword belongs, the fields to be inferred are limited to the lower layers of the field hierarchy dictionary, and the fields are narrowed down as much as possible. A means for eliminating the ambiguity of homonyms used in a field, including a field frequency extraction means for counting the same number of field information for a set of field information to which a keyword belongs, and an aggregated field If there is a parent-child relationship in the field hierarchy in the field hierarchy dictionary, there is a field parent-child relationship extraction means for extracting the parent-child relationship, and there is a sibling relationship in the field hierarchy in the field hierarchy dictionary for the aggregated field The field frequency extraction means and the field sibling relation extraction means for the information output from the field frequency extraction means by using the field sibling relation extraction means for extracting sibling relations Inferring the topic of the input keyword and the set of field information to which the keyword belongs by using the field information analysis means that weights each field using information and performs statistical processing Adopt a field reasoning feature characterized by.

  Finally, in the present invention, a means for generating an appropriate response sentence to a topic with respect to linguistic information transmitted by a person, the keywords extracted by the keyword extracting means with respect to the linguistic information transmitted by the person, and the number thereof , And the number of topics inferred by the field reasoning means, the knowledge of the field parent-child relationship and the field sibling relation used in the reasoning by the field reasoning means, and the number of parameters as parameters. We adopt response sentence construction means characterized by generating a response sentence that does not make the user feel more uncomfortable by combining or transforming sentence templates.

  As described above, the present invention is the head of abbreviations and English notations that combine a plurality of words into one by performing a Web search on the Internet for conversation sentences on unspecified topics such as bulletin boards, blogs, and chats on the Internet. For the abbreviations that connect the characters, as well as the popular names and new words, the topic can be flexibly handled without maintenance of the field association word dictionary or the field hierarchy dictionary, and the topic is appropriate. It is possible to generate a response sentence.

  Furthermore, by extracting parent-child relationships and sibling relationships in fields extracted from language information sent by humans, it is possible to improve the accuracy of topics acquired, and consider parent-child relationships and sibling relationships in fields. A response sentence can be generated.

  This gives the user a sense of security that the language information he / she has sent has been accurately understood, leading to further dialogue.

  The automatic response sentence generation system of the present invention receives language information transmitted by human writing or speaking as an input, extracts keywords from the language information, searches Web pages on the Internet using the extracted keywords, and performs a search. Keywords are also extracted from the results. Then, these keywords are collated with a field classification dictionary storing field names and words belonging to the field, field information is given to the keywords, and humans write or utterances based on these keywords and the field information provided. By doing so, the topic area of the language information transmitted is specified. Based on the identified topic area, the sentence template of the response sentence that seems to be appropriate is selected from the response sentence dictionary, and the keyword and field information are embedded in the sentence template. This is a system that generates an appropriate response sentence.

  The present invention will be described below with reference to the drawings showing the embodiments. [Fig. 1] is a diagram showing an overview of the processing flow for the automatic response sentence generation system of the present invention.

  First, language information sent by human writing or speaking, such as conversation sentences on unspecified topics such as bulletin boards, blogs and chats on the Internet, is converted into text information that can be processed by a computer using the language information input means 1 It is done.

  The text information that can be processed by the computer by the language information input means 1 is extracted by the keyword extraction means 2 from the word that is considered important for identifying the topic of the text information in the text information. Specifically, the sentence is first segmented into words by software called morphological analysis, and part-of-speech information of each word is given, and mainly nouns and proper nouns are extracted as keywords. Various researches have been conducted on this process, for example, [Non-Patent Document 2] and [Non-Patent Document 3] academic papers have been published. The keyword extraction means 2 in the present invention uses the keyword extraction process that is generally performed in this way.

  The keywords extracted from the text information by the keyword extracting means 2 are sent to the Web keyword complementing means 3, and the Web on the Internet is searched using the keywords extracted by the keyword extracting means 2, and hit pages are collected. Additional keywords are obtained by performing keyword extraction processing on the text information of the collected pages.

  Details of this process are shown in Fig. 2. With respect to the keywords extracted from the text information by the keyword extraction means 2, the Web search means 31 searches the Web pages on the Internet and collects the text information of the searched Web pages. Thereafter, keyword extraction processing is performed on the text information of all Web pages collected by the Web keyword aggregating unit 32, and field information is assigned to all extracted keywords by collating with the field associative word dictionary 5. , To collect the information.

In this process, language information that users enter on bulletin boards, blogs, chats, etc. on the Internet includes abbreviations that combine multiple words into abbreviations, acronyms that combine English initials, and common names. Documents that are frequently used to describe documents are often used, such as words extracted from documents using keyword extraction technology, existing national and synonym dictionaries, and a thesaurus that expresses the semantic hierarchical structure between words. This solves the problem that there are many cases where collation is not possible. This is because the words used in the linguistic information input by the user are generally used at that time, and are not registered in dictionaries, but are frequently used on the Web on the Internet. This is because there are many cases.
For example, the word “Giza” such as “Giza cute” is a word that does not exist in 2000, and of course is a word that is not registered in the 2009 dictionary. However, it is frequently used on the Web, and in the 2009 dictionary, “Giza” = “very” is a word that most people know. Furthermore, “about 40 years old” is referred to as “around forty”, and there is the term “all four” for short. “Alafor” is a word that many people can know, but recently, the term “Arakan” is frequently used to cross the age of nearly 60 years with “60th birthday”. The present invention has a feature that it is possible to cope with words that are not listed in the dictionary such as temporary words by adopting the Web keyword complementing means 3.

  The keyword extracted from the text information by the keyword extracting means 2 and the keyword supplemented by the Web keyword complementing means 3 are collated with the information stored in the field associative word dictionary 5 by the field information acquiring means 4, and each keyword is identified. Give information about whether it belongs to such a field.

  As shown in FIG. 4, the field association word dictionary 5 stores a field name and a word belonging to the field in association with each other.

  The keywords extracted from the text information by the keyword extraction means 2 that is the output of the field information acquisition means 4, the keywords supplemented by the Web keyword complementing means 3, and the field information given to each keyword are as one aggregate. Sent to the field reasoning means 6. In the field inference means 6, as shown in FIG. 3, first, the field frequency extraction means 61 aggregates the keywords sent from the field information acquisition means 4 and the field information for each field. The form of the aggregated information is information such as “Furniture (29)” as shown in FIG. This means that there are 29 keywords belonging to the field of “furniture”.

  Next, the field information gathered by the field frequency extracting means 61 is collated with the field hierarchical structure dictionary 7. The contents of the field hierarchical structure dictionary 7 are information indicating that the field names are related in the hierarchical structure as shown in FIG. For example, it shows that <SF>, <Japanese film>, <foreign film>, <movie production>, <animation>, <TV>, etc. exist in the lower layer for the field of <video>.

  Based on the field hierarchical structure dictionary 7, it is determined whether there is a field having a parent-child relationship relationship using the field parent-child relationship extracting unit 62 for the field information collected by the field frequency extracting unit 61. At the same time, the field sibling relation extraction means 63 is used to determine whether there is a field having sibling relations. Here, the field parent-child relationship is the field name of the upper hierarchy and the field name of the lower hierarchy in the field hierarchical structure dictionary 7, and in the example shown in FIG. 6, <video> and <animation> correspond to the parent-child relationship. A sibling relationship is a field in a parallel lower layer belonging to the same upper layer in the field hierarchy dictionary 7. In the example shown in FIG. 6, <Japanese film> and <Western film> correspond to the brother relationship.

  In the field information analysis means 64, the field information aggregated by the field frequency extraction means 61, the parent-child relationship of the field extracted using the field parent-child relationship extraction means 62, and the sibling relation of the field extracted using the field sibling relation extraction means 63. The field information is analyzed using the information of. In the present invention, a procedure for determining a field more accurately is adopted by merging fields in a parent-child relationship or sibling relationship among the field information aggregated by the field frequency extraction means 61.

  Fig. 5 illustrates the fusion of fields that have a parent-child relationship. The field information gathered by the field frequency extracting means 61 is “furniture (29), transportation means (25), chest / cupboards (11), automobile (10), RV car (9)”. Here, “Chance and cupboards (11)” and “Furniture (29)” have a parent-child relationship, and “Automobile (10)” and “RV car (9)” have a parent-child relationship. Therefore, the parent field is merged with the child field. Specifically, the number of keywords belonging to the parent field is added to the keywords belonging to the child field, and the number of keywords is added to the number of keywords in the child field. This fusion is called forward fusion. As a result of the forward fusion, the field information is “Chance / cupboards (40), transportation (25), RV car (19)”. Furthermore, since there is a parent-child relationship between "transportation (25)" and "RV car (19)", they are also fused. Here, the reverse fusion process is shown in which the number of keywords belonging to the child field is added to the number of keywords belonging to the parent field, and that number is added to the number of keywords in the parent field. As a result of this reverse fusion processing, it becomes “Chance, cupboards (40), transportation (44)”.

  The initial field information is “Furniture (29), Transportation (25), Chance and cupboards (11), Automobile (10), RV car (9)”, and “Furniture (29 ) ", But the result of the field integration was" Chance and cupboards (40), transportation means (44) ", and" transportation means (44) "has the largest number. In this way, since keywords in similar fields, such as parent-child relationships and sibling relationships, appear in many fields, the field in which the most keywords appear when judged in a single field It is possible to avoid misrecognition that it is a topic of language information input by the user.

The response sentence construction unit 8 acquires the field information acquired by the field and the field information acquisition unit 4, the keyword extraction unit 2, and the Web keyword complementing unit 3 for the response sentence sentence template stored in the response sentence dictionary 9. Generated by embedding keywords. At this time, the sentence template of the response sentence stored in the response sentence dictionary 9 is selected in consideration of the topic of the linguistic information input by the user inferred by the field reasoning means 6, and the keyword to be embedded is selected.
FIG. 7 shows an example of a sentence template of a response sentence stored in the response sentence dictionary 9. The sentence template of the response sentence is described as tagged text in xml format. Here is an example of a tag.

  <Pat> is a tag that classifies response sentence template patterns by the number of major information according to the number of fields and the topic of language information entered by humans. This is to cope with the difference in the number and location of response information to be embedded in the response sentence template, depending on the number of fields and the topic of language information entered by humans. Fig. 9 shows seven patterns as examples of response sentence template patterns. By generating response sentences corresponding to the number of topics acquired from the number of fields acquired and the input language information in this way, a response sentence that does not make the user feel more uncomfortable is generated. In the figure, <Key> is a tag for embedding an input keyword, and <Field> is a tag for embedding a field.

  In <response>, one response sentence is described in the tag, and the conditions can be classified by the ratio of the first field and the presence / absence of sibling fields and parent / child fields. For example, if <response ratio_up = “70”>, a template with a ratio of the first field of 70% or more can be created, and if <response near = “1”>, one or more sibling fields of the first field exist You can create a template for Conditioning by <response> can combine these three conditions, and can be classified into various cases.

  The <key> tag is a tag for inserting the extracted keyword into the response sentence template. For example, if the keyword “cold” is given to the sentence template “<key>?”, A response sentence “Is it a cold?” Is generated.

  The <field> tag is a tag for inserting the extracted field into the response sentence template. For example, if <field num = “0”>, the first field is inserted into the response sentence template. If the value of element num is not set, a random selection is made from the extracted fields. For example, if the keyword “RV car” is given to the sentence template “<key> is <field> if it is <key>”, the “field” of “automobile” is extracted from the keyword RV car. A response sentence is generated saying "If it is a car, it is a car."

  The <select> tag has a function of randomly selecting a response sentence using a random number from those specified by the <set> tag in the tag. This avoids the situation where the same response sentence is always generated and the user gets bored.

  Here, the follow-up of response sentence generation will be described by taking the case where the keyword “red cliff” is input to the sentence template of the response sentence shown in FIG. First, the keyword “Red Cliff” is obtained by the field reasoning means 6 as a field of “Western painting”. First, “key” is followed by “red cliff” followed by either “to say” or “ha” by <select>, followed by “field” of <field>, and finally <select> Followed by the words “I wonder !!” or “I only have it!”. The response sentence generated by this is “Red Cliff is a Western painting!”, “Red Cliff is a Western painting!”, “Red Cliff is only a Western painting!”, “Red Cliff is a Western painting” One of the four response sentences is generated.

  With the above, it becomes a system that generates a response sentence that does not get tired of the user because the linguistic information input by the user takes the topical field into consideration and is not always the same response sentence.

  The automatic response sentence generation system of the present invention is a system that operates on a computer in an environment in which Web on the Internet can be browsed and in a state in which language information transmitted by human writing or speaking can be accepted. In other words, it can be applied to various bulletin board systems, such as Web bulletin boards, blogs, and chats on computers, and it can be used to temporarily kill users in an environment where processing that increases waiting time is executed. There is also a usage method that provides a simple service. It can also be applied to various guidance systems.

Is a diagram showing an outline of the processing flow of the automatic response sentence generation system of the present invention. These are diagrams showing an outline of the processing flow of the Web keyword complementing means of the present invention. Is a diagram showing an outline of the processing flow of the field reasoning means of the present invention. Fig. 4 is a diagram showing an example of correspondence between a field and a word registered in the field associative word dictionary of the present invention. Fig. 5 is a diagram showing an example of fusion processing in the field of the present invention. FIG. 4 is a diagram showing an example of a field hierarchical structure registered in the field hierarchical dictionary of the present invention. Is a diagram showing an example of a response sentence registered in the response sentence dictionary of the present invention. These are diagrams showing an example of processing of the response sentence construction means of the present invention. Is a diagram showing a variation example of response sentences registered in the response sentence dictionary of the present invention.

DESCRIPTION OF SYMBOLS 1 Language information input means 2 Keyword extraction means 3 Web keyword complementing means 31 Web search means 32 Web keyword aggregation means 4 Field information acquisition means 5 Field association word dictionary 6 Field reasoning means 61 Field frequency extraction means 62 Field parent-child relation extraction means 63 Field Sibling relation extraction means 64 Field information analysis means 7 Field hierarchical structure dictionary 8 Response sentence construction means 9 Response sentence dictionary

Claims (4)

  Humans such as systems that input conversation texts on unspecified topics such as bulletin boards, blogs, and chats on the Internet, guidance systems for operating and using various devices, guidance systems for various exhibition venues, and companies In the environment where the machine or system responds to the language information sent or written by the user, the language information input means for taking in the language information sent by the human and the keyword that becomes the information for determining the field from the language information Keyword extracting means for extracting words, Web keyword complementing means for collecting information on the Internet for language information sent by humans and supplementing keywords, and a field association word dictionary storing correspondence information between keywords and field names The keyword is used for language information sent by humans. Field information acquisition means for acquiring field information to which each keyword belongs by collating the keyword extracted by the extraction means with the keyword complemented by the Web keyword complementing means and the field associative dictionary, and a field hierarchy storing a hierarchical structure between the fields Inferring and outputting the field by collating the structure dictionary and the field information to which the keyword extracted by the keyword extraction means and the keyword complemented by the Web keyword complementing means with respect to the language information transmitted by humans is matched with the field hierarchical structure dictionary Response sentence dictionary containing field inference means, response sentence dictionary storing sentence templates that form response sentences, keywords extracted by keyword extraction means for language information sent by humans, and topics inferred by field inference means By using a response sentence constructing means that connects sentence templates that form the response sentence stored in There automatic answering sentence generating system and generates an appropriate response sentence for outgoing language information for the topic even unspecified topic.   Corresponds to commonly used words that are not registered in existing dictionaries, such as abbreviations that combine multiple words into one, acronyms that combine English initials, and national dictionaries such as popular names. Web search means for increasing the number of words serving as information to be searched, searching for a Web page on the Internet using a keyword extracted by keyword extraction means for language information transmitted by a human, and Web search The keyword information is supplemented with respect to the language information transmitted by the person by using the Web keyword aggregation means for extracting the keywords by the keyword extraction means and aggregating the extracted keywords with respect to the linguistic information obtained by the means. Web keyword complementing means characterized by   In order to infer the field to which this set belongs from the set of keywords and the field information to which the keyword belongs, the field to be inferred is limited to the lower hierarchy of the field hierarchy dictionary, and the fields are narrowed down as much as possible and used in different fields. A means for eliminating the ambiguity of the homonyms that are generated, a field frequency extracting means for counting the same number of field information for a set of field information to which the keyword belongs, and a field hierarchy for the aggregated fields Field parent-child relationship extraction means for extracting a parent-child relationship when there is a parent-child relationship in the field hierarchy in the structure dictionary, and siblings if there is a sibling relationship in the field hierarchy in the field hierarchy structure dictionary for the aggregated field By using the field sibling relation extracting means for extracting the relationship, the information of the field frequency extracting means and the field sibling relation extracting means is obtained for the information output from the field frequency extracting means. It is characterized by inferring the topic of the input keyword and the set of field information to which the keyword belongs by using a field information analysis means that performs statistical processing and weights each field. Field reasoning means.   A means for generating an appropriate response sentence to the topic of linguistic information sent by humans, and for the linguistic information sent by humans, the keywords extracted by the keyword extracting means, the number of them, and the reasoning means By combining the topic and number of topics, knowledge of the field parent-child relationship and the field sibling relationship used when inferring with the field reasoning means, and the number of parameters as parameters, combining the sentence templates that form the response sentence stored in the response sentence dictionary A response sentence constructing means that generates a response sentence that does not make the user feel more uncomfortable by transforming.

Images