KR20130021944A - Method and apparatus for descriptive question answering - Google Patents
Method and apparatus for descriptive question answering Download PDFInfo
- Publication number
- KR20130021944A KR20130021944A KR1020110084528A KR20110084528A KR20130021944A KR 20130021944 A KR20130021944 A KR 20130021944A KR 1020110084528 A KR1020110084528 A KR 1020110084528A KR 20110084528 A KR20110084528 A KR 20110084528A KR 20130021944 A KR20130021944 A KR 20130021944A
- Authority
- KR
- South Korea
- Prior art keywords
- question
- user
- correct answer
- search
- answer
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 239000000284 extract Substances 0.000 claims abstract description 11
- 238000004458 analytical method Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 description 8
- 238000010801 machine learning Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 208000011580 syndromic disease Diseases 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 229940068196 placebo Drugs 0.000 description 2
- 239000000902 placebo Substances 0.000 description 2
- 208000019695 Migraine disease Diseases 0.000 description 1
- 206010027603 Migraine headaches Diseases 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
Landscapes
- Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a descriptive question and answer technique, which extracts key keywords by analyzing a user's natural language question, adds clue words to improve search performance, and searches related documents using a search engine. Search for snippets retrieved through search engines, find and weight documents that contain answers to descriptive questions, rerank the documents with higher weights to rank higher, and find the correct answers within the reranked search results. It is characterized by providing the user with the surrounding information and the correct answer. According to the present invention, it is possible to obtain information desired by a user quickly and simply by automatically presenting a correct answer to a user's narrative question. Thus, the user can handle most of the possible questions that the user is asking by using a familiar natural language or by using existing short-answer questions.
Description
The present invention relates to a question answering technique based on a question answering system, and more particularly, to a method and an apparatus for describing a question answering method suitable for receiving a descriptive question in a natural language form from a user and automatically extracting and providing a correct answer in a sentence form. .
In general, the information retrieval system refers to a system that can easily and quickly retrieve data containing the desired information for a large amount of documents, media, and the like. Documents to be searched for information may be web documents or large documents used by a company.
In this case, the information search receives a combination of keywords such as 'information' and 'search' from the user as a query, and provides a service that finds and provides the most relevant document in the system to the user. The user searches for a document that may have information to find through an information retrieval system, and obtains desired information by reading the retrieved document.
The user's query usually consists of one to five words that can represent the user's information needs. However, it is difficult to completely express the user's desire for information using a small number of words, and thus, it is difficult for the user to obtain satisfactory results. In addition, since the user directly finds the desired information in the searched document, and if there is no desired information in the searched document, the user has to go through a re-searching process, it may take a long time to acquire the information.
The question and answer system is a system that can secure the disadvantages of such information retrieval system. In the Q & A system, instead of a keyword-level query, a natural language query is input. The query in the form of natural language is a form of query used in actual language life such as 'Who is the 14th president of the United States?', Which has the advantage of being familiar to users and expressing their information needs more specifically.
The question and answer system analyzes the query to find out what information the user wants, searches for documents such as web documents or encyclopedias, and provides the user with a Franklin Pierce answer.
That is, while the information retrieval system only searches for documents related to the query, the question and answer system provides an answer to the user's question directly, so that the user can shorten the process of reading the retrieved document and finding the correct answer.
As a conventional technology that provides a service as an intermediate step between such a question and answer and information retrieval, (Patent Literature 1) analyzes a question, extracts a main keyword, analyzes a type, and then retrieves a document using the extracted keyword, Analyzes the summary by question type and presents it to the user.
There is also a conventional technique of analyzing a question for a question and answer, extracting a keyword, searching for a keyword, and finding a correct answer in the searched document. In the case of (Patent Document 2), a method of extracting correct candidates from the searched documents and presenting them to the user was used.
In the question-answering system of (Patent Documents 1 and 2) according to the prior art operating as described above, since the process can be processed only when the question input by the user is a short answer form (Factoid), for example, There is a problem that cannot be processed for a descriptive query such as?
Accordingly, an embodiment of the present invention may provide a method and apparatus for descriptive question and answer that may receive a descriptive question in natural language form from a user and automatically extract a correct answer in sentence form.
In addition, the embodiment of the present invention, after analyzing the user's natural language questions to extract the key keywords, add the clue words for improving the search performance to the key keywords, search for related documents using a search engine, and search engine Find and weight documents that contain answers to descriptive questions for the snippets retrieved through them, rerank the documents with higher weights to rank higher, and then search for the correct answers within the reranked search results. A narrative question answering method and apparatus which can distinguish surrounding information from correct answers and provide them to a user can be provided.
According to an exemplary embodiment of the present invention, a narrative question answering method receives a narrative question, extracts a key keyword of the question through analysis, classifies the question type according to the intention of the question, and then adds a clue word for each type. Searching related documents in the linked web site document or the previously collected document based on the extracted key keywords and clue words, and reranking the documents including the correct answer among the searched documents so that they are ranked in a higher rank. It may include.
The adding process may determine at least one question type among definitions, causes, methods, objectives, and origins, and may remove words other than the message and the verb through morphological analysis.
The re-ranking process may be performed by retrieving a sentence of a document including the key keyword, considering a word included in the sentence, or considering a distance between the word included in the sentence and the key keyword. It may include a process of scoring the probability of including the descriptive correct answer for each snippet of the document based on the machine learning algorithm, and the process of reranking the high scores in the order of priority based on the score for each snippet.
In the re-ranking process, a pattern may be constructed for each question type, and weights assigned to documents matching the constructed pattern may be re-ranked in the order of the weights.
The narrative question answering method may further include extracting a correct answer sentence from the reranked document, outputting the extracted correct answer sentence, or classifying the correct answer sentence with surrounding information other than the correct answer sentence. Can be.
The narrative question answering apparatus according to an embodiment of the present invention includes an input unit that receives a narrative question, extracts key keywords of the question through analysis of the narrative question, classifies question types according to the intention of the question, and then types A question analysis engine unit for adding a clue word to a web search engine, a web search engine unit for searching a related document in a linked web site document or a previously collected document based on the extracted key keywords and clue words, and a correct answer among the searched documents It may include a re-ranking engine unit for re-ranking the included document to be in a higher rank.
The question analysis engine unit may determine at least one question type among definition, cause, method, purpose, and origin, and may remove words other than the message and the verb through morpheme analysis.
The reranking engine unit searches for a sentence of a document including the key keyword, considers a word included in the sentence, or considers each document in consideration of the distance between the word included in the sentence and the key keyword. The probability of including descriptive correct answers for each snippet of is scored based on the machine learning algorithm, and the high scores can be reranked based on the score for each snippet.
In addition, the reranking engine unit may construct a pattern for each question type, assign a weight assigned to a document matching the constructed pattern, and rerank the assigned weights in order of high order.
The narrative question answering method further includes a correct answer extracting unit extracting a correct answer sentence from the reranked document, outputting the extracted correct answer sentence, or classifying the correct answer sentence with surrounding information other than the correct answer sentence. can do.
According to the method and apparatus for a descriptive query response according to the embodiment of the present invention as described above, there are one or more effects as follows.
According to the method and apparatus for a narrative question answering method according to an embodiment of the present invention, a correct answer is automatically presented to a user's narrative question so that information desired by a user can be obtained quickly and simply. Therefore, it is effective for the user to process most of the possible questions that the user asks through the familiar natural language or through the existing short answer question and answer.
1 is a block diagram showing the structure of a descriptive query response device according to an embodiment of the present invention;
2 illustrates an interface for ranking and providing snippets as a result of an information search engine;
3 is a flowchart illustrating an operation procedure of a narrative query response device according to an embodiment of the present invention.
Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.
In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions in the embodiments of the present invention, which may vary depending on the intention of the user, the intention or the custom of the operator. Therefore, the definition should be based on the contents throughout this specification.
Each block of the accompanying block diagrams and combinations of steps of the flowchart may be performed by computer program instructions. These computer program instructions may be loaded into a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus so that the instructions, which may be executed by a processor of a computer or other programmable data processing apparatus, And means for performing the functions described in each step are created. These computer program instructions may be stored in a computer usable or computer readable memory that can be directed to a computer or other programmable data processing equipment to implement functionality in a particular manner, and thus the computer usable or computer readable memory. It is also possible for the instructions stored in to produce an article of manufacture containing instruction means for performing the functions described in each block or flowchart of each step of the block diagram. Computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operating steps may be performed on the computer or other programmable data processing equipment to create a computer-implemented process to create a computer or other programmable data. Instructions that perform processing equipment may also provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.
In addition, each block or step may represent a portion of a module, segment or code that includes one or more executable instructions for executing a specified logical function (s). It should also be noted that in some alternative embodiments, the functions noted in the blocks or steps may occur out of order. For example, the two blocks or steps shown in succession may in fact be executed substantially concurrently or the blocks or steps may sometimes be performed in the reverse order, depending on the functionality involved.
In an embodiment of the present invention, a descriptive question in a natural language form is input from a user and automatically extracts a correct answer in a sentence form, thereby extracting a key keyword by analyzing a user's natural language question and improving search performance on the core keyword. After adding the clue words for the search, search the relevant documents using the search engine, search for the snippets searched through the search engine containing the answers to the descriptive questions, and weighted by weighting After the document is reranked to a high rank, the surrounding information of the correct answer and the correct answer are provided to the user in the reranked search result.
In other words, by analyzing the query of the narrative user and adding the key keyword and the clue keyword, the search performance can be improved, and the user satisfaction can be improved by finding the correct answer in the searched snippet and dividing the surrounding information with the correct answer.
In order to improve search performance, clue words can be added for each type and searched, and the searched documents can be re-ranked using machine learning and pattern matching without using them immediately. And when the correct answer is extracted from the reranked document, the user's satisfaction can be improved by first presenting the correct answer from the high rank to the user.
The present invention may provide the user with the correct answer to the descriptive question, such as 'what is the cause of the earthquake?' In the embodiment of the present invention, by using the existing information retrieval engine for web documents and searching for related documents and extracting the correct answer for the retrieved documents, all general narrative questions that are not domain dependent can be processed. In this case, the narrative question has a feature that the correct answer is provided in a sentence form, unlike the short answer question.
Table 1 below is an example of a descriptive question and a short answer question.
(cause)
(Justice)
(Way)
As shown in Table 1, a question of a descriptive question type may be received as a user input, and a correct answer may be provided to the user in the same form as the information in the 'correct' column of the table.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
1 is a block diagram showing the structure of a descriptive query response device according to an embodiment of the present invention.
Referring to FIG. 1, the narrative
The narrative
In detail, the
The
The question
After determining the question type, stemming analysis can remove words other than words and verbs, and then add clue words appropriate to the question type to improve the search performance. That is, at least one clue word preset for each determined question type may be added.
However, if the question already contains at least one clue word, the clue word may not be additionally included. And when removing words other than spoken and used words, words that appear too often can be removed.
The following (Table 2) shows the type of question, keyword extraction results.
The web
The
For this reranking, there are methods using machine learning and methods using pattern. In the method using the machine learning, the probability of including the descriptive correct answer for each snippet is scored based on the machine learning algorithm such as SVM (Support Vector Machine) using three qualities and exported as a result. The
The first of the three qualities to be used is to extract words from sentences such as key keywords extracted from the question
Secondly, the distance information between the words extracted from the first qualities and the key keywords extracted from the question
Lastly, as the relevance ranking provided by the web
Equation 1 is for calculating a weight in the SVM machine learning method applied in the embodiment of the present invention.
Where x represents the retrieved snippet as a vector using its features. The larger the value of U (x), the higher the probability that the snippet will contain the correct answer. Machine learning learns the appropriate weight value w for each vector over t lessons. Accordingly, the
On the other hand, in the method using a pattern, a pattern is constructed for each descriptive type (for example, a definition type, a method type, a weaning type, etc.), and weights are assigned for each pattern to give a high weight to the snippet in which the high weight pattern appears, How to rank.
Table 3 below shows examples of patterns and weights for each type, which are weighted if there is a matching snippet, and assigned the highest weight for snippets satisfying the various patterns, or assigned to each pattern. A method of summing weights may be used.
The
However, when the user selects or presets the search result through the
3 is a flowchart illustrating an operation procedure of a narrative query response device according to an embodiment of the present invention.
Referring to FIG. 3, in
In
In
The retrieved result is transmitted to the
Thereafter, in
However, when the preset state in the user's selection or descriptive question and answer device is set to "output through extraction of the correct answer from the reranked result" in
As described above, the method and apparatus for descriptive query response according to an embodiment of the present invention receives a descriptive question in a natural language form from a user and automatically extracts a sentence-type correct answer. By adding keywords and clue keywords, the search performance can be improved, and the user satisfaction can be improved by finding the correct answer in the searched snippet and distinguishing the surrounding information from the correct answer.
While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but is capable of various modifications within the scope of the invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the scope of the appended claims, and equivalents thereof.
100: descriptive question and answer device 110: input unit
120: question analysis engine unit 130: web search engine unit
140: reranking engine unit 150: correct answer extraction unit
Claims (1)
Searching related documents in a linked website document or a previously collected document based on the extracted key keywords and clue words;
The process of reranking the documents that contain the correct answer among the retrieved documents to rank higher.
Descriptive query response method comprising a.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020110084528A KR20130021944A (en) | 2011-08-24 | 2011-08-24 | Method and apparatus for descriptive question answering |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020110084528A KR20130021944A (en) | 2011-08-24 | 2011-08-24 | Method and apparatus for descriptive question answering |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20130021944A true KR20130021944A (en) | 2013-03-06 |
Family
ID=48174717
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020110084528A KR20130021944A (en) | 2011-08-24 | 2011-08-24 | Method and apparatus for descriptive question answering |
Country Status (1)
Country | Link |
---|---|
KR (1) | KR20130021944A (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20140128346A (en) * | 2012-02-23 | 2014-11-05 | 도쿠리츠 교세이 호진 죠호 츠신 켄큐 키코 | Non-factoid question answering system and computer program |
KR20160026892A (en) * | 2013-06-27 | 2016-03-09 | 코쿠리츠켄큐카이하츠호진 죠호츠신켄큐키코 | Non-factoid question-and-answer system and method |
KR101667918B1 (en) * | 2015-08-19 | 2016-10-21 | 네이버 주식회사 | Methodand device of providing query-adaptive smart search service |
CN110020009A (en) * | 2017-09-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Online answering method, apparatus and system |
US10460125B2 (en) | 2015-08-27 | 2019-10-29 | Samsung Electronics Co., Ltd. | Apparatus and method for automatic query processing |
KR20220052581A (en) * | 2020-10-21 | 2022-04-28 | 네이버 주식회사 | Method and system for providing search results incorporating the intent of search query |
WO2023063610A1 (en) * | 2021-10-13 | 2023-04-20 | 주식회사 스켈터랩스 | Review analysis system and method using machine reading comprehension |
-
2011
- 2011-08-24 KR KR1020110084528A patent/KR20130021944A/en not_active Application Discontinuation
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20140128346A (en) * | 2012-02-23 | 2014-11-05 | 도쿠리츠 교세이 호진 죠호 츠신 켄큐 키코 | Non-factoid question answering system and computer program |
KR20160026892A (en) * | 2013-06-27 | 2016-03-09 | 코쿠리츠켄큐카이하츠호진 죠호츠신켄큐키코 | Non-factoid question-and-answer system and method |
KR101667918B1 (en) * | 2015-08-19 | 2016-10-21 | 네이버 주식회사 | Methodand device of providing query-adaptive smart search service |
US10460125B2 (en) | 2015-08-27 | 2019-10-29 | Samsung Electronics Co., Ltd. | Apparatus and method for automatic query processing |
CN110020009A (en) * | 2017-09-29 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Online answering method, apparatus and system |
CN110020009B (en) * | 2017-09-29 | 2023-03-21 | 阿里巴巴集团控股有限公司 | Online question and answer method, device and system |
KR20220052581A (en) * | 2020-10-21 | 2022-04-28 | 네이버 주식회사 | Method and system for providing search results incorporating the intent of search query |
WO2023063610A1 (en) * | 2021-10-13 | 2023-04-20 | 주식회사 스켈터랩스 | Review analysis system and method using machine reading comprehension |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20240143679A1 (en) | Search result filters from resource content | |
US9218414B2 (en) | System, method, and user interface for a search engine based on multi-document summarization | |
RU2628200C2 (en) | Supporting guidelines of thematic search | |
KR20130021944A (en) | Method and apparatus for descriptive question answering | |
US6850934B2 (en) | Adaptive search engine query | |
CA2638558C (en) | Topic word generation method and system | |
US8271502B2 (en) | Presenting multiple document summarization with search results | |
US9087084B1 (en) | Feedback enhanced attribute extraction | |
JP2016045652A (en) | Enquiry sentence generation device and computer program | |
KR20100022467A (en) | Cross-language information retrieval | |
US20110307432A1 (en) | Relevance for name segment searches | |
CN106372231A (en) | Search method and device | |
US20120317141A1 (en) | System and method for ordering of semantic sub-keys | |
US9875298B2 (en) | Automatic generation of a search query | |
KR102256007B1 (en) | System and method for searching documents and providing an answer to a natural language question | |
JP2017182646A (en) | Information processing device, program and information processing method | |
JP2009003814A (en) | Method and system for answering question | |
JP2009533767A (en) | System and method for performing a search within a vertical domain | |
US20120023119A1 (en) | Data searching system | |
US20160004697A1 (en) | Bilingual Search Engine for Mobile Devices | |
US8117205B2 (en) | Technique for enhancing a set of website bookmarks by finding related bookmarks based on a latent similarity metric | |
JP4037250B2 (en) | Question answering apparatus, question answering program, and recording medium recording the program | |
CN109284364B (en) | Interactive vocabulary updating method and device for voice microphone-connecting interaction | |
JP2010086210A (en) | Retrieval method, program, and server for preferentially displaying page corresponding to amount of information | |
Kurimo et al. | Morpho challenge evaluation by information retrieval experiments |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WITN | Withdrawal due to no request for examination |