KR20210044697A - Ai based question and answer system and method - Google Patents

Abstract

An AI-based question and answer system and method can present descriptive answers to various questions of the user through the steps of user inquiry, query type classification, question-related document selection, correct answer location search, descriptive answer generation, and answer presentation steps. More specifically, in the question answering system for presenting an AI answer to a user query, a memory for storing instructions, and executing the instructions: classify a user query type of a user query based on a query type classification model, and converts a user query type to a fact query based on the type conversion model, selects a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the fact query query vector, and understands the machine document Based on the machine reading comprehension (MRC) model, the correct answer position is searched inside the document related to the question, and the correct answer derived from the correct answer position is post-processed according to the question type to generate a short answer type answer to the user's query, and a processor configured to present an AI answer by converting a short-answer type answer into an open-ended answer based on the improved Seq2Seq model.

Description

AI-based Q&A system and method {AI BASED QUESTION AND ANSWER SYSTEM AND METHOD}

The present invention relates to an AI-based question-and-answer system and method, and more particularly, to an AI-based question-and-answer system and method that enables narrative answers and answers to various inquiries other than in fact inquiry-type queries.

In recent years, as the automatic Q&A system is widely used, various technologies such as information search for FAQ, ontology construction and information search through SPAQL, and automatic answer generation through machine learning have been researched and developed to build a Q&A system.

Among the recently used methods, the information search method is a method of extracting and indexing keywords, extracting keywords from the query entered by the user, and using it as a search word to find the desired answer. It is ranked by and presented. If there are few prepared Q&As, it is likely that there will be no desired answer, and if there are too many, the search results are large and the user has to search again in the results.

The method using the ontology builds an ontology DB of triple structure based on the prepared Q&A, analyzes the user's query, converts the query to SPAQL, searches the Q&A ontology DB, and provides the answer. Although it has higher accuracy than the information retrieval method, it requires a large amount of ontology input work called knowledge building, which incurs a lot of cost to build the system.

The last method is an artificial neural network (ANN) that learns Q&A and document pairs to create a model, receives the user's query and document as input, and provides the correct answer through the learned model. , Machine Reading Comprehension) technology. This is attracting a lot of attention in that if you learn well, you can find the correct answer to many questions with quite high accuracy without creating an answer in advance. However, although the method using MRC is the most advanced method, it is necessary to find the document with the correct answer in advance and provide it to the MRC engine along with the question, and as a result, only the location of the correct answer in the document is presented as a result. It is difficult, and the MRC technology can only be used for fact-based (Wh-query) questions, so there is a problem that it cannot process other queries (comparative queries, Yes/No queries, etc.).

For the above conventional methods, reference may be made to FIG. 1. In FIG. 1, more detailed contents of an information retrieval method 11 using Q&A, a method 12 using an ontology, and a machine document understanding technology 13 based on machine learning are shown. In order to solve the above-described problems by the conventional methods (11, 12, 13), it may be required to improve an AI-based machine document understanding question-and-answer technique.

Korean Patent Application Publication No. 10-2019-0109656 (2019.09.26)

Accordingly, the technical problem of the present invention is conceived in this respect, and an object of the present invention is to provide an AI-based Q&A system capable of providing a narrative answer and answering various inquiries other than inquiries in fact.

In addition, another object of the present invention is to provide an AI-based question-and-answer method that enables narrative answers and answers to various inquiries other than in fact inquiry-type queries.

The AI-based question-and-answer system and method according to an embodiment for realizing the above-described object of the present invention include user query, query type classification, query-related document selection, search for correct answer location, narrative answer generation, and answer presentation steps. Can provide narrative answers to various inquiries of.

More specifically, a question answering system for presenting an AI answer to a user query according to an embodiment of the present invention includes: a memory storing instructions; And by executing the commands: classifying a user query type of the user query based on a query type classification model, converting the user query type to a fact query based on a query type conversion model, and documenting each of the documents to be searched. Selecting a query-related document from the search target documents based on the similarity between the vector and the query vector of the fact query, and determining the correct answer position within the query-related document based on a Machine Reading Comprehension (MRC) model. Search, and perform post-processing according to the query type for the correct answer derived from the correct answer location to generate a short answer type answer to the user query, and convert the short answer type answer to a narrative answer based on the improved Seq2Seq model to the AI And a processor configured to present an answer.

In one embodiment of the present invention, the query type classification model generates a conditional random field (CRF) training dataset by performing a morpheme analysis on a corpus of queries whose query type is pre-classified, and the CRF training dataset It can be learned in such a way that a classification potential function that classifies a query type of an arbitrary query is formed by applying a CRF machine learning algorithm.

In one embodiment of the present invention, when classifying the user query type, the processor generates user query analysis data by performing morpheme analysis on the user query, and converts the user query analysis data to the classification potential function. By inputting to, the user query type can be classified.

In one embodiment of the present invention, the processor, when converting the user query type to a fact query, if the user query type corresponds to any one of a true/false query, a selection query, and a comparison query, the true/false query Converting a false query into a fact query, or converting the selected query or comparison query into a fact query, and the query type conversion model includes: a first transformation model that converts the true/false query into a fact query, and the selected query or comparison query It may include a second transformation model that converts to a true query.

In an embodiment of the present invention, the first transformation model generates a first transformation learning dataset by performing morpheme analysis on a corpus of true/false queries in which a questionnaire for transformation into a true query is tagged in advance, It may be learned by applying a CRF machine learning algorithm to the first transformation learning dataset to form a first transformation potential function for tagging a questionnaire to an arbitrary true/false query.

In one embodiment of the present invention, the processor generates true/false analysis data by performing morpheme analysis on the true/false query when converting the true/false query into a true query, and generates the true/false analysis data. By inputting false analysis data to the first transformation potential function, the true/false query may be converted into a true query.

In an embodiment of the present invention, the second transformation model performs a morpheme analysis on a corpus of selection queries and comparison queries for which selection/comparison targets for transformation into a true query are previously identified, and the second transformation learning data It can be learned in a manner of generating a set and forming a second transform potential function for identifying a selection/comparison object in an arbitrary selection query or comparison query by applying a CRF machine learning algorithm to the second transformation learning dataset. .

In an embodiment of the present invention, the processor generates selection/comparison analysis data by performing morpheme analysis on the selection query or comparison query when converting the selection query or comparison query into a fact query, The selection/comparison analysis data is input to the second transformation potential function to identify a selection/comparison object in the selection query or comparison query, and the selection query or comparison query as a fact query based on the selection/comparison object Can be converted.

In an embodiment of the present invention, the processor, when selecting the query-related document, is generated in a paragraph unit based on a TF-IDF (Term-Frequency Inverse Document Frequency) weight for each of the search target documents. Cosine similarity between each of the paragraph vectors and the query vector is calculated based on the inverted file index database (DB) of the paragraph vectors, and the query-related document can be selected based on the cosine similarity. have.

In one embodiment of the present invention, when selecting the query-related document, the processor selects a sentence including the keyword from among sentences of the query-related document based on keywords constituting the user query and makes a query. You can create an association summary.

In one embodiment of the present invention, the machine document understanding model includes an encoding layer for quantifying the query-related document and the user query, and an inter-concentration layer for grasping the relationship and context between the query-related document and the user query ( Co-Attention Layer) and an output layer for pointing to a start position and an end position of the correct answer to the user query within the query-related document, and the start according to the LSTM (Long Short-Term Memory) model It can be learned to point to the location and the end location.

In one embodiment of the present invention, when generating the short answer type, the processor, when the user query type is a true/false query, selects yes or no based on the correct answer to the fact query derived from the correct answer position. Generated as the short answer type, and when the user query type is a comparison query or a selection query, one of the selection targets or one of the comparison targets is generated as the short answer type based on the correct answer to the factual query derived from the correct answer location. can do.

In one embodiment of the present invention, the improved Seq2Seq model is based on an encoder structure that outputs state for each time step from morpheme tokens extracted by morpheme analysis of the user query, and the short answer type answer and the state for each time step. Thus, it may include a decoder structure that is learned to generate the narrative answer.

In one embodiment of the present invention, the decoder structure may include a bi-layered Bi-LSTM layer that is output last among the states for each time step and processes the last state corresponding to the ending of the user query. I can.

In one embodiment of the present invention, the processor, when presenting the AI answer, inputs the short answer type answer to the decoder structure to generate a predicted value inferred by the decoder structure, and is inferred by the decoder structure. The narrative answer may be generated by repeating the process of inputting the predicted value back into the decoder structure.

According to another embodiment of the present invention, a query answering method for presenting an AI answer to a user query by a processor includes: classifying a user query type of the user query based on a query type classification model; Converting the user query type into a fact query based on a query type conversion model; Selecting a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the query vector of the fact query; Searching for a location of a correct answer within the query-related document based on a Machine Reading Comprehension (MRC) model; Generating a short answer-type answer to the user query by performing post-processing according to the type of the query on the correct answer derived from the correct answer location; And converting the short answer type answer into a narrative answer based on the improved Seq2Seq model and presenting the AI answer.

In one embodiment of the present invention, the query type classification model generates a conditional random field (CRF) training dataset by performing a morpheme analysis on a corpus of queries whose query type is pre-classified, and the CRF training dataset It can be learned in such a way that a classification potential function that classifies a query type of an arbitrary query is formed by applying a CRF machine learning algorithm.

In one embodiment of the present invention, the step of classifying the user query type,

And generating user query analysis data by performing morpheme analysis on the user query, and classifying the user query type by inputting the user query analysis data into the classification potential function.

In one embodiment of the present invention, the step of converting the user query type to a fact query is to determine the true/false query when the user query type corresponds to any one of a true/false query, a selection query, and a comparison query. Converting the selected query or the comparison query to a fact query, wherein the query type conversion model includes: a first conversion model for converting the true/false query to a fact query, and the selected query or It may include a second transformation model that transforms the comparison query into a fact query.

In an embodiment of the present invention, the first transformation model generates a first transformation learning dataset by performing morpheme analysis on a corpus of true/false queries in which a questionnaire for transformation into a true query is tagged in advance, It may be learned by applying a CRF machine learning algorithm to the first transformation learning dataset to form a first transformation potential function for tagging a questionnaire to an arbitrary true/false query.

In an embodiment of the present invention, the converting of the true/false query into a true query comprises generating true/false analysis data by performing morpheme analysis on the true/false query, and the true/false analysis data. And converting the true/false query into a true query by inputting the first transform potential function to the first transform potential function.

In an embodiment of the present invention, the second transformation model performs a morpheme analysis on a corpus of selection queries and comparison queries for which selection/comparison targets for transformation into a true query are previously identified, and the second transformation learning data It can be learned in a manner of generating a set and forming a second transform potential function for identifying a selection/comparison object in an arbitrary selection query or comparison query by applying a CRF machine learning algorithm to the second transformation learning dataset. .

In one embodiment of the present invention, the converting of the selection query or the comparison query to a fact query comprises generating selection/comparison analysis data by performing a morpheme analysis on the selection query or comparison query, and the selection/comparison Inputting analysis data to the second transformation potential function to identify a selection/comparison object in the selection query or comparison query, and converting the selection query or comparison query into a fact query based on the selection/comparison object It may include.

In one embodiment of the present invention, the step of selecting the query-related documents includes inverse of paragraph vectors generated in units of paragraphs based on a TF-IDF (Term-Frequency Inverse Document Frequency) weight for each of the search target documents. Computing a cosine similarity between each of the paragraph vectors and the query vector based on an Inverted File Index Database (DB), and selecting the query-related document based on the cosine similarity. I can.

In one embodiment of the present invention, the step of selecting the query-related document generates a query-related summary sentence by selecting a sentence including the keyword from among sentences of the query-related document based on keywords constituting the user query. It may include the step of.

In one embodiment of the present invention, the machine document understanding model includes an encoding layer for quantifying the query-related document and the user query, and an inter-concentration layer for grasping the relationship and context between the query-related document and the user query ( Co-Attention Layer) and an output layer for pointing to a start position and an end position of the correct answer to the user query within the query-related document, and the start according to the LSTM (Long Short-Term Memory) model It can be learned to point to the location and the end location.

In one embodiment of the present invention, the step of generating the short-answer answer includes answering yes or no as the short-answer answer based on the correct answer to the fact query derived from the correct answer position when the user query type is a true/false query. Generating and generating any one of the selection targets or any one of the comparison targets as the short answer type based on the correct answer to the fact query derived from the correct answer location when the user query type is a comparison query or a selection query. Can include.

In one embodiment of the present invention, the improved Seq2Seq model is based on an encoder structure that outputs state for each time step from morpheme tokens extracted by morpheme analysis of the user query, and the short answer type answer and the state for each time step. Thus, it may include a decoder structure that is learned to generate the narrative answer.

In one embodiment of the present invention, the decoder structure may include a bi-layered Bi-LSTM layer that is output last among the states for each time step and processes the last state corresponding to the ending of the user query. I can.

In an embodiment of the present invention, the presenting of the AI answer includes generating a predicted value inferred by the decoder structure by inputting the short answer answer to the decoder structure, and re-reducing the predicted value inferred by the decoder structure. It may include the step of generating the narrative answer by repeating the process of inputting into the decoder structure.

According to embodiments of the present invention, the present invention is to solve a practical problem for using AI-based Machine Reading Comprehension (MRC) in the field, and classifies user's questions and provides correct answers to the questions. Documents are selected, the user's question and the selected document are delivered to the MRC engine to find the location of the correct answer, and the derived correct answer can be expressed in narrative form and presented to the user.

More specifically, according to the Q&A system and method that provides AI answers to user queries, user queries such as true/false queries, comparison queries, or selection queries can also be converted into a true query form through query type conversion. However, unlike conventional methods, AI answers may be presented to various types of inquiries.

In addition, the short-answer correct answers searched by the machine document understanding model are not presented, but the descriptive answers converted from the short-answer answers based on the improved Seq2Seq model can be presented. This can be solved, and user convenience of AI Q&A can be improved.

1 is a diagram for explaining conventional methods for implementing an automatic question answering system.
2 is a diagram for explaining an overall process of presenting an AI answer to a user query according to an embodiment of the present invention.
3 is a block diagram showing elements constituting a question answering system for presenting an AI answer to a user query according to an embodiment of the present invention.
4 is a diagram for explaining a process of classifying user query types according to an embodiment of the present invention.
5 is a diagram illustrating a corpus used in a learning process of a query type classification model according to an embodiment of the present invention.
6 is a diagram illustrating a process of converting a true/false query into a true query according to an embodiment of the present invention.
7 is a diagram illustrating a process of converting a comparison query or a selection query into a fact query according to an embodiment of the present invention.
8 is a diagram for explaining a process of selecting a query-related document according to an embodiment of the present invention.
9 is a diagram for explaining a process of selecting a query-related document based on a cosine similarity according to an embodiment of the present invention.
10 is a view for explaining a process of searching for a location of a correct answer within a query-related document based on a machine document understanding (MRC) model according to an embodiment of the present invention.
11 is a view for explaining a process of converting a short answer type answer into a narrative answer based on an improved Seq2Seq model according to an embodiment of the present invention.
12 is a diagram illustrating a process of learning a decoder structure of an improved Seq2Seq model according to an embodiment of the present invention.
13 is a diagram illustrating a process of inferring a narrative answer by a decoder structure of an improved Seq2Seq model according to an embodiment of the present invention.
14 is a flowchart illustrating steps of configuring a question answering method for presenting an AI answer to a user query according to an embodiment of the present invention.

The present invention will be described in detail in the text, since various modifications can be made and various forms can be obtained. However, this is not intended to limit the present invention to a specific form disclosed, it should be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements. Terms such as first and second may be used to describe various elements, but the elements should not be limited by the terms.

The above terms are used only for the purpose of distinguishing one component from another component. The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the present application, terms such as "comprise" or "consist of" are intended to designate the presence of features, numbers, steps, actions, elements, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility of the presence or addition.

Unless otherwise defined, all terms used herein including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

2 is a diagram for explaining an overall process of presenting an AI answer to a user query according to an embodiment of the present invention.

Referring to FIG. 2, a process of generating an answer to a user query in an AI-based machine document understanding (MRC) question-and-answer system is illustrated. The process of generating an answer to a user's query consists of six sub-processes, namely, a query type classification process (21), a query type conversion process (22), a related document selection process (23), a correct answer location search process (24), and It may consist of a post-processing process (short answer-type answer generation process) 25 and a narrative-type answer generation process (26) for each type of query.

In the query type classification process 21, a type of user query may be classified. As shown, the user query may be classified into a fact query, a true/false query, a comparison query, or a selection query, and other types of queries are treated as unprocessable answers, and the question-and-answer process may be terminated.

In the query type conversion process 22, a query type of a user query may be converted into a true query. That is, when a user query is classified as a true/false query, a comparison query, or a selection query in the query type classification process 21, a process of converting the query into a fact query may be performed.

In the related document selection process (23), for a fact query that is classified by the query type classification process (21) or converted by the query type conversion process (22), the document with the greatest correlation therewith can be selected as the query-related document. have. Documents to be searched for which query-related documents are selected may be various types of documents such as business documents, web documents, and e-mails, and may be prepared in an appropriate manner before the question-and-answer process for user queries begins.

In the correct answer location search process 24, a location where the correct answer to the user's query exists may be searched within the query related document selected in the related document selection process 23. The search for the location of the correct answer may be performed using Machine Reading Comprehension (MRC).

In the post-processing process for each type of query (short-answer answer generation process) (25), post-processing is performed for each true/false query, comparison query, or selected query for the correct answer to the user query at the correct answer location searched in the correct answer location search process (24). This may be implemented to generate a short answer type answer to a user query.

In the narrative answer generation process (26), a process of converting the short-answer answers generated in the post-processing process (short-answer generation process) (25) for each type of query into a narrative answer is performed, so that the narrative answer to the user's query is It can be presented as an overall outcome.

Query type classification process (21), query type conversion process (22), related document selection process (23), correct answer location search process (24), post-processing process for each type of query (short answer generation process) (25) and narrative answer generation The AI question-and-answer process consisting of the process 26 may be performed by the question-and-answer system 300 described below.

3 is a block diagram showing elements constituting a question answering system for presenting an AI answer to a user query according to an embodiment of the present invention.

Referring to FIG. 3, a question answering system 300 for presenting an AI answer to a user query may include a memory 310 and a processor 320. However, the present invention is not limited thereto, and other general-purpose elements other than the elements shown in FIG. 3 may be further included in the Q&A system 300.

The Q&A system 300 may be a computing device for presenting an AI answer to a user query. The question-and-answer system 300 may include a memory 310 as a means for storing various data, instructions, at least one program or software, and processes various data by executing instructions or at least one program. The processor 320 may be included as a means for performing the operation.

The memory 310 may store various commands related to presentation of AI answers to user queries in the form of at least one program. For example, the memory 310 may store instructions constituting software such as a computer program or a mobile application. In addition, the memory 310 may store various types of data necessary for execution of an application or program.

The memory 310 may be implemented as a nonvolatile memory such as ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, or volatile memory such as DRAM, SRAM, SDRAM, PRAM, RRAM, FeRAM, etc. Can be implemented as In addition, the memory 310 may be implemented as an HDD, SSD, SD, Micro-SD, or the like.

The processor 320 may present an AI answer to a user query by executing instructions stored in the memory 310. The processor 320 may perform a series of processing steps for implementing AI answer presentation to a user query. In addition, the processor 320 may perform an overall function for controlling the question-and-answer system 300 and may process various operations inside the question-and-answer system 300.

The processor 320 may be implemented as an array of a plurality of logic gates or a general-purpose microprocessor. The processor 320 may be composed of a single processor or a plurality of processors. The processor 320 may be configured integrally with the memory 310 rather than a separate configuration from the memory 310 that stores instructions. For example, the processor 320 may be implemented in the form of at least one of a CPU, a GPU, and an AP provided in the question answering system 300.

The processor 320 may classify the user query type of the user query based on the query type classification model.

In order to answer questions through AI-based machine document understanding (MRC), it may be required that the user query type is actually a query type. Accordingly, a user query type may be classified into one of a fact query, a true/false query, a selection query, a comparison query, and other queries by the query type classification model.

However, if a user query is classified in the form of an instruction, request, command, or description, the question-and-answer process may be terminated because the corresponding form is not the subject of an AI-based machine document understanding (MRC) answer. 4 and 5 to be described later may be referred to for specific details of the user query type classification.

The processor 320 may convert a user query type into a fact query based on the query type conversion model.

When the user query type is any one of a true/false query, a selection query, and a comparison query, the query type can be converted to a fact query by the query type conversion model for the operation of AI-based machine document understanding (MRC). have. 6 and 7 to be described later may be referred to for details of the user query type conversion.

The processor 320 may select a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the query vector of the fact query.

Documents to be searched correspond to candidate documents that provide correct answers to user queries, and the documents to be searched may be numerically converted into a document vector and converted into a database. Based on the similarity between a document vector to be quantified and a query vector of a user query classified or converted in the form of a fact query, a query-related document that is most relevant to the user query among the documents to be searched may be selected. 8 and 9, which will be described later, may be referred to for specific details of selecting a query-related document.

The processor 320 may search for a location of a correct answer within a query-related document based on a Machine Reading Comprehension (MRC) model.

The machine document understanding model may mean an MRC engine that receives a query-related document and a user query in the form of a fact query and outputs a location where an answer to the user query exists in the query-related document. For details of the search for the correct answer location, reference may be made to FIG. 10 to be described later.

The processor 320 may generate a short answer type answer to a user query by performing post-processing according to the type of a question on the correct answer derived from the correct answer location.

Since AI-based machine document understanding (MRC) operates on a true query, if the user query is one of a true/false query, a comparison query, and a selection query, instead of a true query, AI-based machine document understanding ( MRC) results, that is, the correct answer derived from the correct answer position by the machine document understanding model may be subjected to post-processing regarding query type conversion.

For example, if the user query type is a true/false query ("Is Mt. Baekdu at 2744 meters high?"), the correct answer derived from the location of the correct answer is converted from a true/false query ("How many meters is Mt. Baekdu?" ?") can be post-processed (2744 meters), resulting in a short-answer answer (select "Yes" from "Yes" and "No"). Also, in the case of the comparison query and the selection query, post-processing is performed in a manner similar to the case of a true/false query to generate a short answer type answer.

The processor 320 may present an AI answer by converting a short answer type answer into a narrative answer based on the improved Seq2Seq model.

The results of post-processing the correct answers derived by the machine document understanding model correspond to short-answer answers that may cause unkindness to users such as "2744 meters", "Yes" and "Mt. Baekdu", so the corresponding short-answer answers by the improved Seq2Seq model This can be converted into a narrative answer and presented as an AI answer. 11 to 14 to be described later may be referred to for specific contents of the AI answer presentation.

As described above, according to the processor 320, the correct answer by the machine document understanding model can be generated through the query type classification/transformation even for a true/false query, a comparison query, and a selection query that do not correspond to the actual query. It is not limited to queries, and AI questions and answers can be made to more various types of inquiries. Further, according to the processor 320, since a process of converting a short answer type answer to a narrative type answer may be additionally performed, user convenience of the AI question answer may be improved.

4 is a diagram for explaining a process of classifying user query types according to an embodiment of the present invention.

Referring to FIG. 4, a process 41 and a process 42 for generating a query type classification model are illustrated. The generation process 41 may mean a learning process of a query type classification model, and the classification process 42 may mean an inference process of a query type classification model.

The user query can be expressed in various forms according to the requested answer, and the answer to the user query can be naturally provided only when the query type of the user query can be clearly identified. For example, if a user asks "Is the height of Mt. Baekdu 2744m?" and the answer is "2744m", it is not natural. In other words, the user is expecting an answer of "yes" or "no", but if the correct answer is presented as an answer, it will feel a bit wrong. In this case, "Yes, that's right" would be natural.

For the above reasons, it may be important to classify the query types of user queries and provide appropriate answers accordingly. User queries can be classified as follows according to the type of answers the user expects as answers.

In fact, a query is a query that wants real facts as correct answers, such as when, where, who, what, how, and why, and is an AI-based machine document. It is a query that can be expected to be processed in an understanding (MRC), and a form such as a request or command corresponds to a query that is largely irrelevant to the present invention.

For user query type classification, the query type classification model generates a morphological analysis of a corpus of queries whose query types are pre-classified to generate a CRF (Conditional Random Field) training dataset, and a CRF machine for the CRF training dataset. It can be learned by applying a learning algorithm to form a classification potential function that classifies a query type of an arbitrary query.

In the process of generating a query type classification model 41, FIG. 5 may be referred to for details on preparing a corpus, generating a CRF training dataset, and learning to classify a query type.

5 is a diagram illustrating a corpus used in a learning process of a query type classification model according to an embodiment of the present invention.

Referring to FIG. 5, a corpus 50 used in a learning process of a query type classification model is shown. Hereinafter, contents related to the creation of a CRF training dataset and learning to classify a query type may be described.

In the corpus 50, a query type of queries may be pre-classified and prepared for machine learning of a query type classification model. As shown, query types can be prepared in advance with labels of 0 to 5, and among them, a true/false query (0), a selection query (1), a comparison query (2), and a fact query (3) are mainly used. Can be.

For the queries prepared in the corpus 50, morpheme analysis is performed, morphemes may be extracted from the queries, and a part-of-speech feature and query type label may be attached to each morpheme as follows.

As shown in Table 2, when a part-of-speech feature and a query type label are prepared for each morpheme for queries of the corpus 50, the CRF model training may be performed using this as a CRF training dataset. CRF model training may mean a process of training a CRF model with a CRF training dataset by defining a classification potential function. CRF is a conditional probability model and can determine the query type with the highest probability by calculating the probability that a new query belongs to each query type based on the learning result. The classification potential function P(y|x) can be formed as follows.

Equation 1

As described above, when learning about the query type classification model is completed, the mode classification value of the inferred query type label may be classified as the user query type of the user query using the query type classification model. That is, when classifying the user query type, the processor 320 generates user query analysis data by performing morpheme analysis on the user query, and inputs the user query analysis data into the classification potential function to classify the user query type. I can.

When morpheme analysis is performed on the user query and user query analysis data composed of the parts of speech features of each morpheme of the user query is generated, the user query analysis data is input to the classification potential function of the query type classification model, and as a result, the user It can be inferred whether the query type corresponds to a true/false query (0), a selection query (1), a comparison query (2), a fact query (3), and other query types.

6 is a diagram illustrating a process of converting a true/false query into a true query according to an embodiment of the present invention.

Referring to FIG. 6, a first transformation model generation process 61 and a transformation process 62 for converting a true/false query into a true query are illustrated. The generation process 61 may mean a learning process of the first transformation model, and the transformation process 62 may mean an inference process of the first transformation model.

The first transformation model may be included in the query type transformation model together with the second transformation model to be described later. That is, when converting a user query type to a fact query, when the user query type corresponds to any one of a true/false query, a selection query, and a comparison query, the processor 320 converts the true/false query to a fact query, or A selection query or comparison query can be converted to a fact query, and the query type conversion model is a first conversion model that converts a true/false query to a fact query, and a second conversion model that converts the selected or comparison query to a fact query. It may include.

The first transformation model may convert a true/false query into a true query through questionable question tagging. When a questionnaire is tagged to a true/false query, the true/false query can be converted to a true query. For example, if a true/false query such as "Is Mt. Baekdu 2744 meters high?" is tagged with the question "how", in fact the query "How high is Mt. Baekdu?" Can be created.

For the questionnaire tagging as described above, it may be required to prepare a corpus of queries tagged with the questionnaire in advance, and to generate a first transformation learning dataset as in the example of Table 3 below through morpheme analysis for this.

In Table 3, among the morphemes of a true/false query by morpheme analysis, a label is added to a part of "2744 m" that is a questionable subject tagging, and then, a questionable sentence may be tagged to the corresponding part. The label "B_" is added to the morpheme "2744" where the subject of questionable tagging starts, and the label "I_" is added to the morpheme "m" that continues to the tagging target. After that, the questionable word "how" will be tagged. Labels "HOW" may be added to "2744" and "m".

Thereafter, a CRF machine learning algorithm is applied to the generated first transform learning dataset to learn a first transform potential function, and a true/false query is transformed into a true query based on the learned first transform potential function. I can.

That is, when converting a true/false query into a true query, the processor 320 generates true/false analysis data by performing morpheme analysis on the true/false query, and converts the true/false analysis data into a first conversion potential. By entering a function, you can convert a true/false query into a true query. An example of the transformation can be presented as follows.

7 is a diagram illustrating a process of converting a comparison query or a selection query into a fact query according to an embodiment of the present invention.

Referring to FIG. 7, a generation process 71 and a conversion process 72 of a second conversion model for converting a comparison query or a selection query into a fact query are illustrated. The generation process 71 may mean a learning process of the second transformation model, and the transformation process 72 may mean an inference process of the second transformation model.

In the case of an optional query such as "Where is Mt. Baekdu located in South Korea or North Korea?", after finding the correct answer using the fact question "Where is Mt. Baekdu?", we actually inquire about "South Korea" and "North Korea" suggested as the target of choice. Compared with the correct answer to, the answer to the selection query may be presented.

Also, in the case of a comparison query such as "Which is higher, Mount Baekdu or Mount Halla?", "What is the height of Mount Baekdu?" The answer to the comparison query may be presented in a manner that compares the correct answer obtained by separating it into factual queries such as "What is the height of Hallasan?" with the comparison object of the comparison query.

To this end, the second transformation model generates a second transformation learning dataset by performing a morpheme analysis on the corpus of selection queries and comparison queries for which selection/comparison targets for transformation into factual queries are previously identified. 2 Transformation Learning It can be learned by applying a CRF machine learning algorithm to a dataset to form a second transform potential function that identifies a selection/comparison target in an arbitrary selection query or comparison query.

In order to generate the second transformation learning dataset, a corpus may be formed by collecting random selection queries and comparison queries for which a selection/comparison object is previously identified, and a morpheme analysis for the corpus may be performed. As a result of the morpheme analysis, a part-of-speech feature may be identified for each morpheme constituting the selection query or comparison query, and a label may be added to the selection/comparison object as shown in the table below.

As shown in Tables 5 and 6, the part-of-speech quality for each morpheme constituting the query can be analyzed, and a label can be added to the "Middle South and North Korea" section, which is a selection target among the morphemes, Labels may be added to "Mt. Baekdu and Mt. Halla" corresponding to. "B" may be added to the starting point of the comparison/selection object, and "I" may be added to the consecutive morphemes of the comparison/selection object, and "TG" as a target for the comparison/selection object, and the morpheme for listing the objects "JN" may be labeled as "and" and "middle" in the sense of connection.

On the other hand, in Table 6, the morpheme "high" is a part representing the value to be compared in the comparison query, and corresponds to the questionnaire for comparison. "QT" may be labeled. The labeled interrogative sentence for comparative adjectives can then be converted to a questionable noun such as "height?"

When the second transformation learning dataset is generated through morpheme analysis and labeling as shown in Tables 5 and 6, after defining the second transformation potential function that identifies the selection/comparison target in a random selection query or comparison query, Learning of the second transformation model may be completed through CRF learning.

When the training of the second transformation model is completed, an inference process of the second transformation model may be performed using this. That is, when converting the selection query or the comparison query into a fact query, the processor 320 generates selection/comparison analysis data by performing morpheme analysis on the selection query or comparison query, and provides the selection/comparison analysis data. 2 It is possible to identify a selection/comparison object in the selection query or comparison query by inputting it into the transformation potential function, and convert the selection query or the comparison query into a fact query based on the selection/comparison object.

In the case of a selection query, when a selection object is identified in the selection query by inference using the learned second transformation model, a fact query may be generated as shown in the table below in a manner that excludes the syntax of the selection object.

In the case of a comparison query, a fact query can be generated as shown in the table below by converting the comparison adjective interrogative sentence into an interrogative noun in the comparison query by reasoning using the learned second transformation model.

In Table 8, it can be confirmed that the comparative adjective interrogative sentence "is it high?" has been converted to the interrogative noun "how much?" Meanwhile, the conversion of the interrogative sentence of the comparative adjective to the interrogative noun may be performed by referring to a dictionary stored in advance. In the dictionary, when the adjective "high" is used as an interrogative sentence, the point that the interrogative noun used is "height", and that the usage method is "What is the height of ~?" can be stored.

8 is a diagram for explaining a process of selecting a query-related document according to an embodiment of the present invention.

Referring to FIG. 8, a target document indexing process 81 for converting documents to be searched into an inverted file index database (DB) and a related document selection process for selecting a query related document based on an inverted file index DB ( 82) is shown.

In the target document indexing process 81, a process of converting each of the documents to be searched into paragraph vectors of a paragraph unit and converting them into a DB may be performed. Since the documents to be searched can be converted into paragraph vectors that can be numerically analyzed through such a DB process, a query-related document closest to the query vector of the user query can be selected based on this.

In the related document selection process 82, the user query is also quantified as a query vector, and the similarity with the paragraph vectors that are previously quantified and DBized in the target document indexing process 81 can be calculated, and searched based on the similarity value. Some documents or paragraphs among target documents may be selected as query-related documents.

To this end, the processor 320, when selecting a query-related document, indexes inverse files of paragraph vectors generated in paragraph units based on a TF-IDF (Term-Frequency Inverse Document Frequency) weight for each of the documents to be searched. Cosine similarity between each of the paragraph vectors and the query vector may be calculated based on the Inverted File Index Database (DB), and a query related document may be selected based on the cosine similarity.

On the other hand, as the volume of the document increases, it is divided into several topics and subsections, so one document becomes very large as information for obtaining a clue of a question, and thus each document to be searched can be separated and processed in paragraph units. Therefore, in the present invention, the term document may be understood to have the same meaning as the paragraph.

The TF-IDF weight is based on the frequency at which the keywords appear in each search target document (or paragraph unit) after extracting all keywords included in the entire search target document through morpheme analysis of the entire search target document. As, it can be created as follows.

Equation 2

In Equation 2, x may indicate the index of the keyword, y may indicate the index of the document (or paragraph), w _x,y indicates the weight of the keyword x in the document y, and tf _x,y indicates the keyword x in the document y The frequency of occurrence of df _x may be the number of documents (or paragraphs) in which the keyword x exists, and N may be the number of documents (or paragraphs).

_{The document vector d j} for the document j based on the TF-IDF weight calculated as above may be expressed as follows when the indexes of keywords are 1 to t.

Equation 3

Document vectors (paragraph vectors) generated in units of paragraphs as described above may be converted into a DB through an Inverted File Index so that they can be provided in real time in the process of selecting related documents for future queries.

When the inverse file index DB is created, _{the cosine similarity between the paragraph vector d j} provided therefrom and the query vector q of the user query can be calculated as follows. The query vector q can be calculated based on the frequency of occurrence of each keyword in the same way as the process of generating the paragraph vector.

Equation 4

The cosine similarity as in Equation 4 is calculated N times between one query vector q and N paragraph vectors d _j , so that N cosine similarities may be provided. 9 may be referred to for a process of generating N cosine similarities and a specific method of selecting a query-related document using the same.

9 is a diagram for explaining a process of selecting a query-related document based on a cosine similarity according to an embodiment of the present invention.

Referring to FIG. 9, when the number of keywords t is 3 and the total number of paragraphs N is 6, a graph 90 for explaining a process of calculating the cosine similarity between each of the paragraph vectors and the query vector is shown. .

In the graph 90, since the number of keywords t is 3, each of the paragraph vectors and the query vector can be displayed as a three-dimensional vector, and one query vector and 6 are displayed according to the values of the three elements of each vector. A position in a three-dimensional vector space of three paragraph vectors may be displayed.

In this case, six cosine similarity values may be calculated by Equation 4 above, and a paragraph corresponding to a paragraph vector having the largest cosine similarity among the six values may be selected as a query-related document. .

Meanwhile, when selecting a query-related document, the processor 320 may generate a query-related summary by selecting a sentence including the keyword from among sentences of a query-related document based on keywords constituting a user query.

In the AI-based MRC engine, which will be performed after the selection of the query-related documents, when searching for the correct answer location for the entire document of one large size, performance is impaired due to excessive calculations, and a number of noises irrelevant to the query are output. Q&A quality may deteriorate. To solve this problem, after selecting the query-related documents, the sentences containing the most keywords constituting the user query are extracted from the inside of the query-related documents, and the query-related documents are summarized in an appropriate size, so that the quality of MRC and the It may become possible to improve performance.

10 is a view for explaining a process of searching for a location of a correct answer within a query-related document based on a machine document understanding (MRC) model according to an embodiment of the present invention.

Referring to FIG. 10, a machine document understanding (MRC) model may be composed of an encoding layer 101, an interfocus layer 102, and an output layer 103. The machine document understanding (MRC) model may receive a target document (query-related document) and a user query, and output a start position and an end position of a correct answer to the user query in the query-related document.

That is, the machine document understanding model includes an encoding layer 101 for quantifying a query-related document and a user query, and a co-attention layer 102 for grasping the relationship and context between the query-related document and the user query. And an output layer 103 for pointing to a start position and an end position in which the correct answer to the user's query exists in the query-related document, and the start position and end according to the LSTM (Long Short-Term Memory) model. It can be learned to point to a location.

In order to learn a machine document understanding model, a process of vectorizing necessary information in relation to word level, character level, context level, and feature may be enumerated so that necessary information can be quantified and calculated. At the word level, One-Hot, Word2Vec, GloVe, and fastText can be used, at the character level, processing of unknown words can be performed using RNN and CNN, and CoVe, ELMo, etc. can be performed at the context level. Through this, contextual embedding can be calculated, and embeddings in which additional information other than vocabulary is utilized, such as other POS, Named Entity, Dependency Lable, TF, etc., can be utilized as a feature.

In the encoding layer 101, a target document including context information may be input together with a user query, and in this case, the target document may mean a query related document selected in a previous process.

In the inter-concentration layer 102, a vector constituting a user query, a vector constituting a query-related document, information on the location of the correct answer, etc., using data embedded for quantification, and the inside of the user query and the query-related document Relationships between contexts in the context can be grasped, and the vector to be focused on through learning can be suggested.

The output layer 103 may be trained to infer the start and end of a location where the correct answer to the user's query exists in the query-related document based on the result of the inter-concentration layer 102.

As above, when the machine document understanding (MCR) model is trained, an MRC engine that identifies the context in the query-related document through this, and suggests where the correct answer to the user's query exists in the query-related document can be provided. have. In particular, the MRC engine is trained through an LSTM (Long Short-Term Memory) model, and can point the start and end positions of the correct answer to the user query through the Point Network in the later inference process.

On the other hand, after the start and end positions of the location where the correct answer to the user's query exists is pointed within the query-related document by the machine document understanding (MCR) model, the correct answer derived from the corresponding answer location is determined according to the type of the query. Post-processing may be performed, and as a result, short-answer responses to user queries may be generated.

Specifically, when generating a short answer type, the processor 320 may generate yes or no as a short answer type answer based on the correct answer to the fact query derived from the correct answer position when the user query type is a true/false query, and , When the user query type is a comparison query or a selection query, one of the selection targets or one of the comparison targets may be generated as a short answer type based on the correct answer to the fact query derived from the correct answer location.

That is, in the present invention, in addition to the fact query, the true/false query, the selection query, and the comparison query may also be converted into the form of a fact query to generate an answer by the MRC engine, but the answer by the MRC engine is an answer to the fact query. A post-processing process may be required to convert this back into the form of an answer to a true/false query, a selection query, and a comparison query.

For example, when an MRC engine's answer to a true/false query is generated, it can be converted back into the form of an answer to a true/false query such as “Yes” or “No”, and a selection query or a comparison query Even in the case of, the MRC engine's answer may be converted back into a form of an answer to a selection query or a form of an answer to a comparison query. By such post-processing according to the query type, the AI query answer of the present invention can be performed not only for the fact query, but also for the true/false query, the selection query, and the comparison query.

11 is a view for explaining a process of converting a short answer type answer into a narrative answer based on an improved Seq2Seq model according to an embodiment of the present invention.

Referring to FIG. 11, the improved Seq2Seq model may include an encoder structure 111 and a decoder structure 112. The encoder structure 111 is a structure for generating a morpheme token for each time step by encoding a user query, and the decoder structure 112 may be a structure for converting a short answer type answer into a narrative answer based on a morpheme token for each time step. .

Specifically, the improved Seq2Seq model generates a narrative answer based on the encoder structure 111 that outputs the state for each time step from the morpheme tokens extracted by morpheme analysis of the user query, and the short answer answer and the state for each time step. It may include a decoder structure 112 that is learned to be performed.

In particular, since the improved Seq2Seq model does not require the entire sentence, it is assuming that the short answer type answer by the MRC engine (machine document understanding model) already exists, so it may become possible to generate a narrative answer with less computational amount. In addition, since learning and inference can be focused on the ending of a user query, a high-accuracy narrative answer can be provided with a small amount of computation.

In the encoder structure 111, morpheme analysis and lemmatization may be performed in order to grasp the characteristics of the sentence order of the user query to generate morpheme tokens constituting the user query. For example, as shown, "which", "degree", "requires" and "does" correspond to the morpheme tokens, and the morpheme tokens are of the encoder structure 111 according to a time step designated with an appropriate size. It is input to the Bi-LSTM model, and as its output, a state for each time step can be created. The states for each time step may be aggregated as all states to be utilized in the decoder structure 112 later.

Meanwhile, when the user query is smaller than the specified time size, post padding (<PAD>) may be used, and when the user query is larger than the specified time size, a sentence cut by the time size from the end of the sentence may be used. Since the user query is cut from the end rather than from the beginning, even when the user query is longer than the time size, the last state may correspond to the predicate portion located at the end of the user query, that is, the ending portion.

The states generated by the encoder structure 111 can then be utilized to generate a narrative answer in the decoder structure 112. 12 and 13 may be referred to for details thereof.

12 is a diagram illustrating a process of learning a decoder structure of an improved Seq2Seq model according to an embodiment of the present invention.

Referring to FIG. 12, a decoder structure 120 of an improved Seq2Seq model is shown, and the decoder structure 120 may be for explaining a process in which the decoder structure 112 of FIG. 11 is learned.

In the learning process of the decoder structure 120, a short answer type answer by the MRC engine for each time step may be input to the decoder structure 120 in a form in which a narrative tone is combined. In the decoder structure 120, the next word may be predicted for each time step. In this case, the inference result of the previous time step is not used as an input to improve accuracy, and teacher forcing may be used instead.

In particular, the last state generated in the encoder structure 111 may play a key role in the learning process and inference process of the decoder structure 120. That is, the decoder structures 112, 120, and 130 may include a Bi-LSTM layer in the form of a double layer that is output last among the states for each time step and processes the last state corresponding to the ending of a user query.

As described above, the last state corresponds to the predicate part, that is, the ending part, located at the end of the user query, and the ending part may act most importantly to generate a narrative answer. As shown in the figure, the decoder structure 120 ), the last state can be input to the bi-layered Bi-LSTM layer and processed. Meanwhile, in order to extract the correct answer, an affine transformation is performed in the last layer of the decoder structure 120, so that the next word may be learned.

13 is a diagram for explaining a process in which a decoder structure of an improved Seq2Seq model infers a narrative answer according to an embodiment of the present invention.

13, a decoder structure 130 of the improved Seq2Seq model is shown. The decoder structure 130 may be understood as representing a state after the decoder structure 112 of FIG. 11 and the decoder structure 120 of FIG. 12 have completed learning.

The decoder structure 130 converts short-answer answers into narrative answers in a manner in which words to be placed next for each time step are inferred one by one according to the teacher forcing technique by taking short-answer responses by the MRC engine (machine document understanding model) as input values. can do. For example, when the short answer type "15" by the MRC engine is input to the decoder structure 130 as shown, then "minute", "degree", "requirement" and "will" are inferred in sequence, and narrative The answer "It will take about 15 minutes" can be generated.

To this end, when presenting an AI answer, the processor 320 may generate a predicted value inferred by the decoder structure 130 by inputting the short answer type answer into the decoder structure 130, and A narrative answer may be generated by repeating the process of inputting the inferred predicted value back to the decoder structure 130.

According to the above method, in addition to the results of the MRC engine (Machine Document Understanding Model) that provides only short-answer answers, narrative speech that sounds kind without awkward is added, and narrative answers can be provided. An example in which a short answer type answer is converted into a narrative answer by the decoder structure 130 of the improved Seq2Seq model may be referred to in the table below.

14 is a flowchart illustrating steps of configuring a question answering method for presenting an AI answer to a user query according to an embodiment of the present invention.

Referring to FIG. 14, a question answering method for presenting an AI answer to a user query may include steps 141 to 146. However, the present invention is not limited thereto, and general steps other than the steps shown in FIG. 14 may be further included in the Q&A method of FIG. 14.

The question-and-answer method of FIG. 14 may consist of steps processed in a time series by the question-and-answer system 300 described above with reference to FIGS. 2 to 13. Therefore, although the contents of the question-and-answer method of FIG. 14 are omitted below, the contents described above for the question-and-answer system 300 of FIGS. 2 to 13 are the same for the question-and-answer method of FIG. 14. Can be applied.

In step 141, the question answering system 300 may classify the user query type of the user query based on the query type classification model.

The query type classification model generates a CRF (Conditional Random Field) training dataset by performing a morpheme analysis on the corpus of queries whose query types are pre-classified, and applies a CRF machine learning algorithm to the CRF training dataset. It can be learned by forming a classification potential function that classifies the query type of a query.

The query answering system 300 may generate user query analysis data by performing a morpheme analysis on a user query, and classify a user query type by inputting the user query analysis data into a classification potential function.

In step 142, the query answering system 300 may convert the user query type into a true query based on the query type conversion model.

When the user query type corresponds to any one of a true/false query, a selection query, and a comparison query, the question answering system 300 converts a true/false query into a fact query, or converts a selected query or a comparison query into a fact query. The query type conversion model may include a first conversion model for converting a true/false query into a true query and a second conversion model for converting a selection query or a comparison query into a fact query.

The first transformation model generates a first transformation training dataset by performing morpheme analysis on a corpus of true/false queries in which a questionnaire for transformation into a true query is tagged in advance, and CRF for the first transformation training dataset. It may be learned by applying a machine learning algorithm to form a first transform potential function that tags a questionable question in an arbitrary true/false query.

The question-and-answer system 300 may generate true/false analysis data by performing morpheme analysis on true/false queries, and input the true/false analysis data to the first transformation potential function to provide true/false queries. In fact, it can be converted to a query.

The second transformation model generates a second transformation training dataset by performing morpheme analysis on the corpus of selection queries and comparison queries for which selection/comparison targets for transformation into queries are previously identified, and the second transformation training data It may be learned by applying a CRF machine learning algorithm for the set to form a second transform potential function that identifies a selection/comparison object in an arbitrary selection query or comparison query.

The question-and-answer system 300 may generate selection/comparison analysis data by performing morpheme analysis on a selection query or a comparison query, and input selection/comparison analysis data to a second transformation potential function to make a selection query or a comparison query The selection/comparison object in can be identified, and the selection query or the comparison query can be converted into a fact query based on the selection/comparison object.

In step 143, the query answering system 300 may select a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the query vector of the fact query.

The question-and-answer system 300 is based on an inverted file index database (DB) of paragraph vectors generated for each paragraph based on a TF-IDF (Term-Frequency Inverse Document Frequency) weight for each of the documents to be searched. Accordingly, a cosine similarity between each of the paragraph vectors and a query vector can be calculated, and a query-related document can be selected based on the cosine similarity.

The query answering system 300 may generate a query-related summary by selecting a sentence including the keyword from among sentences of a query-related document based on keywords constituting a user query.

In step 144, the question-and-answer system 300 may search for a correct answer position within the query-related document based on a Machine Reading Comprehension (MRC) model.

The machine document understanding model includes an encoding layer for quantifying query-related documents and user queries, a co-attention layer for grasping the relationship and context between the query-related documents and user queries, and the user within the query-related documents. It may include an output layer for pointing a start position and an end position where a correct answer to a query exists, and may be trained to point a start position and an end position according to a Long Short-Term Memory (LSTM) model.

In step 145, the Q&A system 300 may generate a short answer type answer to a user query by performing post-processing according to the type of a question on the correct answer derived from the correct answer location.

The question-and-answer system 300 may generate yes or no as a short answer type answer based on the correct answer to the fact query derived from the correct answer position when the user query type is a true/false query, and the user query type is a comparison query or In the case of a selection query, one of the selection targets or one of the comparison targets may be generated as a short answer type based on the correct answer to the fact query derived from the correct answer location.

In step 146, the Q&A system 300 may present an AI answer by converting a short answer type answer into a narrative answer based on the improved Seq2Seq model.

The improved Seq2Seq model includes an encoder structure that outputs state for each time step from morpheme tokens extracted by morpheme analysis of a user query, and a decoder structure that is learned to generate a narrative answer based on a short answer and a state for each time step. Can include.

The decoder structure may include a Bi-LSTM layer in the form of a double layer that is output last among the states for each time step and processes the last state corresponding to the ending of a user query.

The Q&A system 300 may generate a prediction value inferred by the decoder structure by inputting a short answer type answer into the decoder structure, and repeat the process of inputting the prediction value inferred by the decoder structure back into the decoder structure to provide a narrative answer. Can be generated.

Meanwhile, the question-and-answer method of FIG. 14 may be recorded in a computer-readable recording medium in which at least one program or software including instructions for executing the method is recorded.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like may be included. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter as well as machine language codes such as those produced by a compiler.

Although the above has been described with reference to the embodiments, those skilled in the art can variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that there is.

Claims (16)

In a question-and-answer system that presents AI answers to user inquiries,
A memory for storing instructions; And
By executing the above commands:
Classify the user query type of the user query based on the query type classification model,
Convert the user query type to a fact query based on the query type conversion model,
Selecting a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the query vector of the fact query,
Based on a Machine Reading Comprehension (MRC) model, search for the location of the correct answer within the query-related document,
For the correct answer derived from the correct answer location, post-processing according to the query type is performed to generate a short answer type answer to the user query,
And a processor configured to present the AI answer by converting the short answer answer into a narrative answer based on the improved Seq2Seq model.
The method of claim 1,
The query type classification model,
A morpheme analysis of the corpus of queries with pre-classified query types is performed to create a CRF (Conditional Random Field) training dataset,
A question answering system that is learned by forming a classification potential function for classifying a query type of an arbitrary query by applying a CRF machine learning algorithm to the CRF learning dataset.
The method of claim 2,
The processor, when classifying the user query type,
Generate user query analysis data by performing morpheme analysis on the user query,
The query answering system for classifying the user query type by inputting the user query analysis data into the classification potential function.
The method of claim 1,
When the processor converts the user query type into a fact query,
When the user query type corresponds to any one of a true/false query, a selection query, and a comparison query, converting the true/false query into a fact query or converting the selected query or comparison query into a fact query,
The query type conversion model includes a first conversion model for converting the true/false query to a fact query and a second conversion model for converting the selected query or comparison query to a fact query.
The method of claim 4,
The first transformation model,
In fact, a first transformation learning dataset is created by performing a morpheme analysis on the corpus of true/false queries tagged with a questionnaire for transformation into a query in advance,
A question-answering system that is learned by applying a CRF machine learning algorithm to the first transformed learning dataset to form a first transformed potential function for tagging a questionnaire to an arbitrary true/false query.
The method of claim 5,
When the processor converts the true/false query into a true query,
To generate true/false analysis data by performing morpheme analysis on the true/false query,
And converting the true/false query into a true query by inputting the true/false analysis data into the first transformation potential function.
The method of claim 4,
The second transformation model,
In fact, a second transformation training dataset is created by performing a morpheme analysis on the corpus of selection queries and comparison queries for which the selection/comparison target for transformation into a query is previously identified, and
A question-answering system that is learned by applying a CRF machine learning algorithm to the second transform learning dataset to form a second transform potential function that identifies a selection/comparison object in an arbitrary selection query or a comparison query.
The method of claim 7,
When the processor converts the selection query or the comparison query into a fact query,
To generate selection/comparison analysis data by performing morpheme analysis on the selection query or comparison query,
Inputting the selection/comparison analysis data to the second transformation potential function to identify a selection/comparison target in the selection query or comparison query,
A question answering system for converting the selection query or the comparison query into a fact query based on the selection/comparison object.
The method of claim 1,
When the processor selects the query-related document,
Each of the paragraph vectors based on an inverted file index database (DB) of paragraph vectors generated for each paragraph based on a TF-IDF (Term-Frequency Inverse Document Frequency) weight for each of the search target documents, and Calculate the cosine similarity between the query vectors,
A question answering system for selecting the query-related document based on the cosine similarity.
The method of claim 1,
When the processor selects the query-related document,
A query-response system for generating a query-related summary by selecting a sentence including the keyword from among sentences of the query-related document based on keywords constituting the user query.
The method of claim 1,
The machine document understanding model,
An encoding layer for quantifying the query-related document and the user query, a co-attention layer for identifying the relationship and context between the query-related document and the user query, and the user within the query-related document Includes an output layer for pointing a start position and an end position where the correct answer to the query exists,
The question-and-answer system, which is learned to point the start position and end position according to a Long Short-Term Memory (LSTM) model.
The method of claim 1,
The processor, when generating the short answer type answer,
If the user query type is a true/false query, yes or no is generated as the short answer type answer based on the correct answer to the fact query derived from the correct answer location,
When the user query type is a comparison query or a selection query, one of the selection targets or one of the comparison targets is generated as the short answer type answer based on the correct answer to the fact query derived from the correct answer location. The method of claim 1,
The improved Seq2Seq model,
An encoder structure for outputting a state for each time step from morpheme tokens extracted by morpheme analysis of the user query, and
And a decoder structure that is learned to generate the narrative answer based on the short answer type and the state for each time step.
The method of claim 13,
The decoder structure includes a Bi-LSTM layer in the form of a double layer that is output last among the states for each time step and processes a last state corresponding to a ending of the user query.
The method of claim 13,
The processor, when presenting the AI answer,
Inputting the short answer answer to the decoder structure to generate a predicted value inferred by the decoder structure,
The question-and-answer system for generating the narrative answer by repeating the process of inputting the predicted value inferred by the decoder structure back into the decoder structure.
In the question-and-answer method of presenting an AI answer to a user query by a processor,
Classifying a user query type of the user query based on a query type classification model;
Converting the user query type into a fact query based on a query type conversion model;
Selecting a query-related document from the search target documents based on the similarity between the document vector of each of the search target documents and the query vector of the fact query;
Searching for a location of a correct answer within the query-related document based on a Machine Reading Comprehension (MRC) model;
Generating a short answer-type answer to the user query by performing post-processing according to the type of the query on the correct answer derived from the correct answer location; And
Comprising the step of presenting the AI answer by converting the short answer type answer into a narrative answer based on the improved Seq2Seq model.

Images