KR100726176B1 - Method and apparatus for extracting correct answer in question answering system - Google Patents

Abstract

A method and a device for extracting a correct answer in a Q/A(question/answer) system are provided to improve performance of the Q/A system by constructing various heterogeneous distributed information sources and extracting the desired answer from the information source storing the most suitable answer to the information need of the user through various correct answer extracting techniques. A language analyzer(130) linguistically analyzes a sentence of a target document(110) or a question sentence(120) of the user. A heterogeneous correct answer indexer(150) constructs the heterogeneous distributed information sources(140) by indexing the correct answer through various correct answer indexing techniques according to the sentence of the target document or the question sentence of the user. A multi-correct answer extractor(160) extracts the most suitable candidate answers to the information need of the user from the information sources through the correct answer extracting techniques. A correct answer manager(170) infers the extracted candidate answers according to the information need of the user, and integrates and presents the correct answers to the user.

Description

Method and apparatus for extracting correct answer in question answering system}

1 is a block diagram of an apparatus for extracting multiple correct answers according to the present invention.

FIG. 2 is a diagram for explaining a heterogeneous correct indexer in FIG. 1.

FIG. 3 is a diagram for describing a multiple answer extractor in FIG. 1.

4 is a flowchart of a multiple correct answer extraction method according to the present invention.

5 is a detailed flowchart of a language analysis step of FIG. 4.

6 is a view showing an example of the correct answer type in the present invention.

FIG. 7 is a detailed flowchart of a multiple correct answer extraction step in FIG. 4.

8 is a detailed flowchart of a correct answer management step in FIG. 4.

Explanation of symbols on the main parts of the drawings

110: Target Document 120: Question Sentence

130: language analyzer 140: heterogeneous distributed information source

150: heterogeneous correct indexer

151: pattern-based correct indexer 152: statistics-based correct indexer

153: Short Answer Indexer 154: Descriptive Answer Indexer

155: enumerated correct indexer

160: Multiple Answer Extractor

161: pattern based answer extractor 162: statistical based answer extractor

163: Short Answer Answer Extractor 164: Descriptive Answer Extractor

165: enumerated correct answer extractor

170: Answer Manager

The present invention relates to a method and apparatus for extracting multiple correct answers in a question and answer system, and in particular, to grasp a user's information need and access a variety of information sources in which a desired answer is stored to extract a correct answer. The present invention relates to a method and apparatus for extracting a correct answer.

The Q & A system searches and presents the correct answers that can be the answer to the user's query in the document set and presents them to the user. It extracts only the information corresponding to the user's query from a myriad of information on the website and provides it to the user. The question and answer retrieval method is widely used.

As one of such question and answer search methods, Korean Patent Registration No. 446288 (2004.05.25) only constructs a candidate correct answer that can be formulated in relational DB structure and FAQ list in advance and then builds an existing information retrieval system. A method of searching for related paragraphs and estimating the closest data based on pattern and language analysis among them is disclosed.

However, the question and answer retrieval method can process only very simple questions because candidate candidate answers are limited to candidates that can be formalized into DB entries, and use existing information retrieval systems to retrieve responses to user queries. This causes a problem of slow system response or manual labor.

As another question answer search method, the research paper (In Question Answering, Two Heads Are Better Than One, Proceedings of HLT-NAACL 2003, P24 ~ 31, 2003.6) contains several answer extraction agents to find answers from various sources. After extracting the correct answer from the weighted weight (weight) to the correct answer is extracted accordingly according to the ranking of the correct answer and presenting the correct answer of the first among them is disclosed a method that can provide a more accurate correct answer to the user.

However, since the question and answer search method always takes the correct answer for every agent and presents the first order answer regardless of the user's question characteristics, when the simple question is processed, the search speed is performed by performing an unnecessary extraction process. In addition to slowing down, there is a problem that the user cannot extract the correct answer that the user wants in the case of a complex question of not finding the correct answer with one agent.

In addition, the question and answer search methods have a problem in that the correct answer to the user's question is simply searched by referring to a searched DB for searching, and therefore, a long time is required to extract the correct answer, and thus the question and answer time is very long.

The present invention has been made to solve the above problems, an object of the present invention is to build a variety of heterogeneous distributed information sources for question and answer and to access the information source that stores the correct answer that best matches the user's intention It is to provide a multi-correct answer extraction method and apparatus that can improve the query response performance by extracting the correct answer desired by the user through a variety of correct answer extraction techniques.

In order to achieve the above object, the multiple correct answer extraction method according to the present invention comprises: (a) a language analysis step of linguistically analyzing a sentence of a target document or a user's question sentence; (b) constructing heterogeneous distributed information sources through various correct answer indexing techniques according to the sentence of the target document or the user's question characteristic; (c) a multiple correct answer extracting step of extracting multiple candidate correct answers based on the heterogeneous distributed information source according to a user's question intention; And (d) a correct answer management step of integrating candidate correct answers multiplexed by the step (c) according to the user's question intent.

On the other hand, in order to achieve the above object, the multiple correct answer extraction apparatus according to the present invention, a language analyzer for linguistically analyzing the sentence of the target document or the user's question sentence; A heterogeneous correct answer indexer for constructing a heterogeneous distributed information source by performing a correct answer index through various correct answer indexing techniques according to a sentence of the target document or a user's question characteristic; A multiple correct answer extractor which extracts multiple candidate correct answers that best match a user's question intent through various correct answer extraction techniques based on the heterogeneous distributed information source; And a correct answer manager for inferring and integrating the extracted candidate correct answers according to a user's question intention and presenting them to the user.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an apparatus for extracting multiple correct answers according to the present invention.

As shown in FIG. 1, the multi-correct answer extracting apparatus 100 according to the present invention includes a language analyzer 130 for linguistically analyzing a sentence in a target document 110 or a user's question sentence 120. The heterogeneous correct indexer 150 constructs the heterogeneous distributed information source 140 by indexing the correct answer through various correct indexing techniques according to the characteristics of the candidate correct answer, and the heterogeneous variance in which the candidate correct answer is stored by identifying the user's question intent. Multiple answer extractor 160 for accessing the information source 140 to extract the correct answer in multiple, and the correct answer manager 170 that infers and integrates the extracted correct answer according to the user's question intentions to present to the user.

The language analyzer 130 receives a sentence in the target document 110 or a question sentence 120 of the user and analyzes the input sentence linguistically to answer the type of the correct sentence according to a predefined classification item. The language analysis method used herein will be described in detail with reference to FIGS. 5 and 6.

The heterogeneous correct indexer 150 is based on the results analyzed by the language analyzer 130 according to the characteristics of candidate correct answers existing in the sentence or question sentence 120 of the target document 110, the different types of heterogeneous correct indexer The heterogeneous distributed information source 140 is used to construct the heterogeneous distributed information source 140. Hereinafter, the heterogeneous correct indexer 150 will be described in more detail with reference to FIG. 2.

FIG. 2 is a diagram for explaining the heterogeneous correct indexer 150 in FIG. 1.

As shown in FIG. 2, the heterogeneous correct indexer 150 includes a pattern-based correct indexer 151, a statistics-based correct indexer 152, a short-answer correct indexer 153, a descriptive correct indexer 154, and an enumerated correct indexer 155. It consists of).

The pattern-based correct answer indexer 151 performs a function of indexing the correct answer recognized as a pattern or a rule from the result analyzed by the language analyzer 130, and the statistical-based correct indexer 152 is a candidate through a machine learning technique. Learn the qualities of correct answers and index the correct answers based on them.

The pattern-based correct indexer 151 and the statistical-based correct indexer 152 index only the correct answer corresponding to a predetermined characteristic, and candidate correct answers belonging to other types may be indexed through the short-answer correct indexer 153. desirable.

On the other hand, the user asked questions such as "Why is blood red?", "Which country is Niagara Falls in?" As described above, there are cases where descriptive correct answers or enumerated correct answers are required in addition to the short answers. In this case, the correct answers to the questions are preferably processed by the descriptive correct indexer 154 and the enumerated correct indexer 155, respectively.

Here is an example of the correct answer handled by each indexer:

(1) Pattern-based correct answer indexer (151)

-Target sentence: "The longest river in the world is the Nile"

Candidate Correct: Nile

(2) statistics-based correct answer indexer (152)

-Subject sentence: Park Jung-hee was born in Sunsan, Gyeongbuk.

-Candidate Correct Answer: (Park Chung Hee, Hometown) = (Gyeongbuk Sunsan)

(3) Short answer type indexer (153)

-Target sentence: The International Red Cross prepares the Nightingale Award and selects and commends excellent nurses from around the world every year.

Candidates' answers: International Red Cross (ORG), Nightingale Award (PRIZE), Nurse (OCCUPATION)

(4) Descriptive Answer Indexer (154)

Subject sentence: Red color due to hemoglobin in blood.

Candidate Correct Answer: (cause of blood-red) = (because of hemoglobin in the blood)

(5) enumerated correct indexer (155)

Target sentence: Niagara Falls is on the border between the United States and Canada.

-Candidate Correct Answer: USA, Canada

Referring back to FIG. 1, the multiple correct answer extractor 160 selects a correct answer extractor suitable for a user's question intent and difficulty, and extracts multiple candidate correct answers based on the heterogeneous distributed information source 140. Hereinafter, the multiple answer extractor 160 will be described in more detail with reference to FIG. 3.

FIG. 3 is a diagram for describing the multiple correct answer extractor 160 in FIG. 1.

As shown in FIG. 3, the multiple answer extractor 160 includes a pattern-based correct answer extractor 161, a statistical-based correct answer extractor 162, a short answer correct answer extractor 163, a descriptive correct answer extractor 164, and an enumerated correct answer extractor 165. ), Each correct answer extractor extracts the correct answer desired by the user through various correct answer extraction techniques by accessing the heterogeneous distributed information source 140 that stores the correct answer that best matches the user's question intent.

Since the form of the candidate correct answer obtained through the multiple correct extractor 160 is almost the same as the form of the candidate correct answer obtained through the heterogeneous correct indexer 150, a detailed description thereof will be omitted.

Referring back to FIG. 1, the correct answer manager 170 infers and integrates the correct answer extracted through the multiple correct extractor 160 according to the user's question intent and presents the result to the user.

Hereinafter, a method for extracting multiple correct answers according to the present invention will be described in detail with reference to the accompanying drawings.

4 is a flowchart of a multiple correct answer extraction method according to the present invention.

As shown in FIG. 4, the multi-correct answer extraction method according to the present invention includes a language analysis step (S410) of largely linguistic analysis of a user's question sentence and various correct answer indexing techniques according to characteristics of candidate correct answers. Index heterogeneous answer index step (S420) to build a heterogeneous distributed information source 140 by indexing, and grasp the intention of the user's question to access the heterogeneous distributed information source 140 that stores candidate correct answers to extract the correct answer multiple times Multiple correct answer extraction step (S430), and the correct answer can be divided into the correct answer management step (S440) of inferring and integrating the user's question intention to present to the user, each step will be described in more detail as follows.

[1] Language Analysis Step (S410)

The language analysis step S410 is a step of classifying questions by linguistically analyzing a sentence in the target document 110 or a question sentence 120 of the user. Hereinafter, the language analysis step S410 will be described with reference to FIGS. 5 and 6. ) In more detail.

FIG. 5 is a detailed flowchart of the language analysis step S410 of FIG. 4, and FIG. 6 is a diagram illustrating an example of a correct answer type in the present invention.

Referring to FIG. 5, in the morphological analysis step (S411), the input sentence is decomposed into morphological units to determine parts of speech of each morpheme, and in the correct answer type recognition step (S412), the 160 semantic classifications defined in advance are correct answer types. In this case, the correct answer type is life name, academic name, theory, artifact, organization name, place name, culture / civilization, date, time, quantity, event, animal, plant, It is classified in detail by field such as substance.

Next, in the lexical semantic tagging step (S413), a concept is assigned to each noun using a noun vocabulary concept network, and in the lexical semantic determining step (S414), based on the information tagged through the lexical semantic tagging step (S413). Determine the vocabulary meaning for each vocabulary.

Next, in the parsing step (S415), the syntax structure for each sentence is analyzed and outputted, and in the LF (Logical Form) extraction step (S416), the LF structure is extracted by reconstructing the parsing result using the syntax around the word. do.

[2] heterogeneous answer index step (S420)

The heterogeneous answer index step (S420) is heterogeneous distributed information source 140 through the heterogeneous correct answer indexer 150 according to the characteristics of the candidate correct answers existing in the sentence in the target document 110 or the user's question sentence 120. In the step of constructing, since the correct answer index through the heterogeneous correct indexer 150 has been described in detail with reference to FIG. 2, a detailed description thereof will be omitted.

[3] Extracting multiple correct answers (S430)

The multi-correct answer extracting step (S430) is a step of extracting a candidate correct answer based on the heterogeneous distributed information source 140 by selecting a correct answer extractor suitable for the user's question intention and difficulty. Hereinafter, the multi-correct answer extracting step is described with reference to FIG. 7. It will be described in more detail with respect to (S430).

FIG. 7 is a detailed flowchart of the multiple-correct answer extraction step S430 of FIG. 4.

First, after grasping the user's question intention and difficulty based on the user's question analysis result analyzed through the language analysis step (S410) (S431), the correct answer extractor suitable for the user's question intention and difficulty is selected by the multi-strategy configuration. (S432).

Here, a multi-strategy construct means what kind of correct answer extractor should be selected to extract the correct answer that matches the user question, which one or more correct answer extractors should be used in multiple, what type of candidate correct answers should be integrated, etc. Comprising the overall rules for extracting the correct answer, the multi-strategy composition includes the rule structure of the user question, the choice of the correct answer extractor according to the key word, the clue vocabulary, the correct answer type, and the integration rule of the candidate correct answers.

That is, when the correct answer extractor suitable for the user's question intention and difficulty is selected by the multi-strategy configuration as described above, the candidate correct answer most suitable for the user question is extracted from the heterogeneous distributed information source 140 through the selected correct answer extractor. (S433).

[4] answer management step (S440)

The correct answer management step S440 is a step of inferring and integrating the correct answer extracted through the multiple correct answer extracting step S430 according to the user's intention of the question and presenting it to the user. It will be described in more detail with respect to (S440).

FIG. 8 is a detailed flowchart of the correct answer management step S440 in FIG. 4.

As shown in FIG. 8, in the correct answer management step S440, the candidate correct answer extracted through the multiple correct answer extracting step S430 may be integrated according to the correct answer type for the user question (S441). The method may be divided into a comparison of relevance and correct reranking (S442) for comparing and determining and re-ranking whether or not it is suitable for a user's question intention, and presenting the integrated candidate correct answer and additional information (S443).

On the other hand, the following example is an example showing the process of analyzing the question intention of the user in accordance with the present invention and extracting the correct answer based on this to finally present the correct answer to the user.

Yes)

User Question: How long is the longest river in the world?

(1) Identifying user's question intent

-Answer Type: Length (QUANTITY: QT_LENGTH)

-Required Information: 'World's Longest River' + 'Long River'

(2) Multiple Method Answer Extraction Step

-Pattern based answer extractor: 'World's Longest River'

Statistical-based Answer Extractor: the length of the river

(3) integration of correct answers

Candidate answers for pattern-based correct answer extractors: Nile (longest river in the world)

Candidate answers for statistical-based correct answer extractors: 6690 km (length of Nile)

(4) Goodness-of-fit comparison and correct reranking steps

-Goodness of fit between candidate correct answers and user questions

(5) Presentation of Candidate Answers and Additional Information

Candidate Correct Answer: 6690 km

-Additional Information: Location of the Nile, photos of the Nile, and other information related to the Nile

As described above, the heterogeneous distributed information source is constructed through the heterogeneous correct indexer according to various characteristics of the candidate correct answers, and after grasping the intention of the user's question, the correct answer extractor is selected to extract the candidate correct answer multiple times, and the extracted candidate By integrating and presenting the correct answers according to the types of correct answers to user questions, the performance of information retrieval and question answering systems can be improved.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, etc.), an optical reading medium (for example, a CD-ROM, DVD, etc.) and a carrier wave (for example, the Internet). Storage medium).

So far, the present invention has been described with reference to the preferred embodiments, and those skilled in the art to which the present invention belongs may be embodied in a modified form without departing from the essential characteristics of the present invention. You will understand. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

As described above, according to the present invention, various heterogeneous distributed information sources are constructed for question and answer, and the user accesses the information source that stores the correct answer that best fits the user's intention of the question, and the user wants the user through the various answer extraction techniques. Since only the correct answer can be extracted, there is an effect that can improve the performance of the information retrieval and question answering system.

In addition, according to the present invention, according to the difficulty of the question by presenting the correct answer based on multiple strategies to select a single correct extraction or parallel correct answer extraction, there is an effect that can improve the user response speed.

Claims (14)

(a) a language analysis step of linguistically analyzing a sentence of a target document or a question sentence of a user; (b) constructing heterogeneous distributed information sources through various correct answer indexing techniques according to the sentence of the target document or the user's question characteristic; (c) a multiple correct answer extracting step of extracting multiple candidate correct answers based on the heterogeneous distributed information source according to a user's question intention; And and (d) a correct answer management step of integrating candidate correct answers multiplexed by the step (c) according to a user's question intent. The method of claim 1, wherein step (a) comprises: A morphological analysis step of decomposing the user's question sentence into morpheme units and determining parts of speech of each morpheme; Recognizing a correct answer type for a user's question according to a predefined classification item; A vocabulary meaning tagging step of tagging each vocabulary meaning using a noun vocabulary concept network; A vocabulary meaning determining step of determining a vocabulary meaning for each vocabulary based on the information tagged through the vocabulary meaning tagging step; A parsing step of analyzing a syntax structure for each sentence; And And a LF extraction step of extracting a logical form (LF) structure by reconstructing the result of parsing using the framework around the verb. According to claim 1, wherein step (b), Selecting a correct answer indexing technique suitable for a characteristic of a candidate correct answer included in a sentence of the target document or a question sentence of a user; And And constructing the heterogeneous distributed information source according to the characteristics of the candidate correct answer through the selected correct answer indexing technique. The method of claim 3, wherein the answer indexing technique is: A method for extracting multiple answers in a question-and-response system comprising at least one of a pattern-based correct index technique, a statistical-based correct index technique, a short-answer correct index technique, a descriptive correct index technique, and an enumerated correct index technique. The method of claim 1, wherein step (c) comprises: Selecting a correct answer extraction method suitable for a user's question intent and difficulty by constructing a multiple strategy; And And extracting multiple candidate correct answers based on the heterogeneous distributed information source through the selected correct answer extracting technique. The method of claim 5, wherein the multi-strategy configuration, A method for extracting multiple answers in a question-and-answer system, comprising a rule for selecting a correct answer extraction technique suitable for a user's question intent and difficulty, and an integrated rule for the multiple candidate candidates. The method of claim 5, wherein the correct answer extraction technique, A method for extracting multiple answers in a question-and-response system comprising at least one of a pattern-based correct answer extraction technique, a statistical-based correct answer extracting technique, a short answer correcting extracting technique, a descriptive correct extracting technique, and an enumerated correct extracting technique. The method of claim 1, wherein step (d) Integrating candidate correct answers multiplexed by the step (c) according to the correct answer type for the user question; Comparing and reranking whether the integrated correct answer is suitable for a user's question intent; And And presenting the integrated candidate correct answer and additional information. A language analyzer for linguistically analyzing a sentence of a target document or a question sentence of a user; A heterogeneous correct answer indexer for constructing a heterogeneous distributed information source by performing a correct answer index through various correct answer indexing techniques according to a sentence of the target document or a user's question characteristic; A multiple correct answer extractor which extracts multiple candidate correct answers that best match a user's question intent through various correct answer extraction techniques based on the heterogeneous distributed information source; And And a correct answer manager for inferring and integrating the extracted candidate correct answers according to a user's question intention and presenting them to a user. The language analyzer of claim 9, wherein the language analyzer comprises: After recognizing the correct answer type for the sentence of the target document or the user's question sentence according to a predefined classification item, the syntax structure is analyzed based on the semantic meaning of each vocabulary, and the LF (Logical Form) structure is extracted according to the parse result. Multiple answer extracting device in the question and answer system, characterized in that. The method of claim 9, wherein the heterogeneous correct indexer, At least one of a pattern-based correct indexer, a statistical-based correct indexer, a short answer correct indexer, a descriptive correct indexer, and an enumerated correct indexer, The apparatus for extracting multiple answers in a question-and-answer system according to at least one of the correct answer indexers according to the characteristics of candidate correct answers included in the sentence of the target document or the user's question sentence. The method of claim 9, wherein the multiple correct answer extractor, At least one of a pattern-based correct answer extractor, a statistical-based correct extractor, a short answer correct extractor, a descriptive correct extractor, and an enumerated correct extractor, Multiple answer extraction apparatus in a question-and-answer system, characterized in that the correct answer extractor of at least one of the correct answer extractor extracts the candidate answer that best matches the user's question intent according to the user's question intent and difficulty by the multi-strategy configuration . The method of claim 12, wherein the multi-strategy configuration, An apparatus for extracting multiple answers in a question-and-answer system comprising a rule for selecting a correct answer extractor suitable for a user's question intent and difficulty, and an integrated rule for the multiple candidate candidates. The correct answer manager according to claim 9 or 13, Integrating the extracted candidate correct answers according to the type of the correct answer to the user's question by the multi-strategy configuration, then comparing and re-ranking the proposed correct answers to the user's question intention and presenting them to the user. An apparatus for extracting multiple correct answers in a question and answer system.

Images