KR102355591B1 - Method and apparatus of generating question-answer learning model through reinforcement learning - Google Patents

Abstract

The present invention relates to a method of operating a question-and-answer model through reinforcement learning in an apparatus for generating a response to a query, comprising: sampling a latent variable in an arbitrary paragraph in a first agent; extracting a data set of questions and answers from the paragraphs based on the latent variables; determining, in the second agent, whether to apply the extracted query and response data set to training of a question-and-answer model that generates a response to an arbitrary query; and applying the performance change value of the question-and-answer model to the first agent and the second agent as a reward.

Description

Method and device for generating a question-and-answer learning model through reinforcement learning {METHOD AND APPARATUS OF GENERATING QUESTION-ANSWER LEARNING MODEL THROUGH REINFORCEMENT LEARNING}

The present invention relates to a method and apparatus for generating a data set for improving the performance of a QA model that generates a response to a query. More specifically, the present invention relates to a method of extracting a question-and-answer pair from an arbitrary context, but extracting question-and-answer learning data so that a QA model can learn the question-and-answer pair to improve performance.

Due to the recent development of technology, various question-and-answer systems using deep learning technology have been developed. Recently, when a user assigns a question to the system in natural language, a system has been demonstrated that automatically provides an answer to the question using a question-and-answer (QA) model. However, this deep learning-based question and answer (QA) model eventually requires a large amount of learning data, so it had no choice but to automatically respond to some specific fields or topics. Furthermore, there was a problem in that a considerable amount of time and money was required to construct the training data required to reach a certain level or higher of reliability of the question-and-answer (QA) model.

On the other hand, in the case of a recent question-and-answer learning model, since pre-training is performed using a language model objective in a large corpus, the performance is already quite high. Therefore, the performance of the Q&A learning model does not improve linearly by simply using more data for learning. Therefore, it is becoming very important to secure efficient sample data to improve machine learning performance.

Registered Patent Publication No. 10-1605430, 2016.03.22.

An object of the present invention is to provide a method for efficiently extracting training data of a QA model that generates a response to a query from an arbitrary context. Furthermore, the present invention provides a method of generating a training data set that efficiently improves the performance of the QA model by giving the performance of the updated QA model as a reward to a model that performs a function of extracting data for a query and response pair. aim to

In addition, the problem to be solved by the present invention is different from the existing question-and-answer (QA) model, in which the user had to manually collect and learn the joint distribution of question and response pairs, based on the inputted paragraph even if no response candidate group is given. To provide a question-and-answer generation model that simultaneously generates a question-and-answer data set that can appear as

In the apparatus for generating a response to a query according to an embodiment of the present invention, a method of operating a question-and-answer model through reinforcement learning includes, in a first agent, sampling a latent variable in an arbitrary paragraph; extracting a data set of questions and answers from the paragraphs based on the latent variables; determining, in the second agent, whether to apply the extracted query and response data set to training of a question-and-answer model that generates a response to an arbitrary query; and applying the performance change value of the Q&A model to the first agent and the second agent as a reward.

Furthermore, a method for generating a question-and-answer learning model through adversarial learning according to an aspect of the present invention for solving the above-described problem includes: sampling a latent variable based on a constraint in an input paragraph; generating a response based on the latent variable; generating a query based on the response; and machine-learning a question-and-answer learning model using the generated question-and-answer data set, wherein the constraint is that the latent variable increases the loss of the question-and-answer learning model while increasing the data diversity. (manifold) can be controlled to exist.

In addition, the sampling of the latent variable may include: generating a hidden representation for each word in an input paragraph and storing it in the memory; and sampling a latent variable based on the hidden expression.

In addition, generating the response may include: generating a weighted sum vector by calculating the importance of the hidden expression stored in the memory through attention using the sampled latent variable as a query; and generating a response span based on the latent variable and the weighted sum vector.

In addition, the inputted paragraph may structure the infobox data extracted from the crawled data on the website.

In addition, when the infobox data is not extracted from the crawled data, a standardized paragraph based on the data frequency in the table recognized through column recognition may be input as the paragraph.

The method may further include evaluating the quality of the machine translation by evaluating the accuracy of an answer based on the inputted paragraph being data on which machine translation has been completed, and based on the generated query and answer data set.

An apparatus for generating a question-and-answer learning model through adversarial learning according to another aspect of the present invention includes: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation, wherein the operation performed by the one or more processors is performed according to constraints in an input paragraph. ) for sampling a latent variable based on; an operation for generating a response based on the latent variable; an operation for generating a query based on the response; and machine learning the question-and-answer learning model using the generated query and response data set.

In addition, the constraint may be controlled so that the latent variable exists in a data manifold while increasing the loss of the Q&A learning model.

In addition, the operation of sampling the latent variable may include an operation of generating a hidden representation for each word in the input paragraph and storing it in a memory; and an operation of sampling a latent variable based on the hidden expression.

In addition, the operation for generating the response may include: an operation of generating a weighted sum vector by calculating the importance of the hidden expression stored in the memory through attention using the sampled latent variable as a query; and an operation of generating a span based on the latent variable and the weighted sum vector.

Also, according to another embodiment, a program stored in a medium may be included in combination with a computer, which is hardware, to generate a question-and-answer data set based on a paragraph input.

Other specific details of the invention are included in the detailed description and drawings.

According to an embodiment of the present invention, since a data set of questions and responses for training QA model is automatically generated from an arbitrary context, it is possible to sample several pairs of questions and responses with only one paragraph input. It is possible to build more diverse data than the model of In addition, the performance of the Q&A learning model can be improved by machine learning the constructed data.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1A is a conceptual diagram schematically illustrating a machine learning Q&A learning model according to an embodiment.
1B is a conceptual diagram schematically illustrating a question-and-answer learning model through adversarial learning according to an embodiment.
2 is a diagram for explaining a method for improving the performance of a question and answer model according to an embodiment.
3A is a flowchart illustrating a method of extracting a Q&A dataset from a paragraph and learning a QA model.
3B is a flowchart illustrating a method of generating a question-and-answer data set based on a paragraph input according to an exemplary embodiment.
4 is a conceptual diagram schematically illustrating a question-and-answer dataset generation model based on a paragraph input.
5 is a diagram for explaining an example of a method of generating a question and answer dataset in a paragraph according to an embodiment.
6 is a diagram for explaining a method of generating a question and answer data set in a standardized infobox according to an embodiment.
7 is a diagram for explaining a method of generating a question-and-answer data set from an unstructured page according to an embodiment.
8 is a block diagram schematically illustrating an internal configuration of an apparatus for generating a question-and-answer data set according to an embodiment.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only those of ordinary skill in the art to which the present invention pertains, to complete the disclosure of the present invention. It is provided to fully understand the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as "beneath" or "beneath" of another component is placed "above" of the other component. can get Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating a question-and-answer learning model through machine learning according to an embodiment.

Referring to FIG. 1A , a question-and-answer learning model 100 using reinforcement learning according to an embodiment of the present invention includes a decoder 140 that performs a function of extracting a question-and-answer set from a paragraph, and a response to an arbitrary query. The first agent 122 that extracts the latent variable from the QA model 170 and the paragraph that performs the provided function, and the second agent that determines whether to apply the response and query data set extracted from the paragraph to the QA model 170 (160).

More specifically, when a paragraph 101 is input, the first agent 122 may generate a latent variable 103 from the context of the paragraph. The latent variable 103 includes processing the contexts included in the paragraph 101 and expressing the paragraph in a vector form.

When the latent variable 103 is input to the decoder 140 , the decoder 140 may generate a data set of questions and answers based on the latent variable 103 .

The second agent 160 may determine whether to apply the generated data set of the query and response to the learning of the QA model 170 .

There are two problems in using the query and response pair generated by the decoder 140 , that is, the QA pair, for training the QA model 170 . First, since most of the generated QA pairs are similar to the data already trained by the QA model, there is a risk of overfitting when the QA model 170 is trained by adding the generated QA pairs. Second, since noise is present in the generated QA pair, there is a risk of degrading the performance of the QA model if the noise is not refined.

Therefore, according to an embodiment of the present invention, the second agent 160 may determine whether to apply the QA pair generated by the decoder 140 to the learning of the QA model 170 . That is, according to the embodiment of the present invention, since the second agent leaves only a QA pair capable of improving the performance of the QA model and filters the remaining data with noise, the above-described noise problems can be solved.

Furthermore, according to an embodiment of the present invention, the first agent 122 does not generate a latent variable in the context according to a predefined rule, but generates a latent variable helpful in learning the QA model. It corresponds to a machine learning model that learns to do so. Therefore, since the first agent forms a latent variable space that can increase the performance of the QA model, the above-mentioned overfitting problem can be solved.

Thereafter, the QA model 170 may be updated by learning the QA pair provided by the second agent 160 .

On the other hand, according to the embodiment of the present invention, the first agent 122 is a machine learning model that performs a function of generating the latent variable 103 by processing the contexts included in the arbitrary paragraph 101 and expressing it in a vector form. to be. Furthermore, the second agent 160 is a machine learning model that performs a function of determining whether to apply the generated query and response data set to the learning of the QA model 170 .

That is, the first agent creates a latent variable from a paragraph in order to generate a QA pair necessary for training the QA model, and the second agent determines whether to apply the generated QA pair to the training of the QA model, so that the first agent The performance of 122 and the second agent 160 will be directly related to the performance of the QA model 170 .

Therefore, the QA learning model 100 according to the embodiment of the present invention is updated to increase the performance of the first agent model 122, the second agent model 160, and the QA model 170 by organically connecting them. The performance of the model is given as a reward to the first agent model 122 and the second agent model 160 to reinforce-learning the first agent model and the second agent model. According to this, the question-and-answer learning model 100 has the effect of generating a training data set that efficiently increases the performance of the QA model 170 .

Referring back to FIG. 1A , the above-described reinforcement learning-based question-and-answer learning model 100 may have the objective function of Equation 1 below.

를 최대화하는 파라미터

를 추정할 수 있다.Here, x, y, c, and z correspond to the context and latent variables of questions, responses, and paragraphs, respectively. Using this, the objective function

parameters that maximize

can be estimated.

The objective function of Equation 1 is for learning a model capable of generating a QA pair, and is not directly related to reinforcement learning of the question-and-answer learning model 100 . Reinforcement learning is to improve the performance of the first agent model 122 and the second agent model 160 .

The first agent model and the second agent model of FIG. 1A may have the objective function of Equation 2 below.

Here, the F1 score refers to the performance or accuracy of the QA model, and corresponds to a value calculated by how high accuracy the QA model can extract a response to an arbitrary query. According to Equation 2, since the performance of the QA model is applied as a reward to the learning of the first agent and the second agent, the performance of the QA model will be learned in a direction to increase. That is, the first agent will be trained to extract latent variables that can improve the performance of the QA model from the paragraph, and the second agent will leave only the data set that can improve the performance of the QA model from the QA pair generated by the decoder, and the rest of the noise is You will learn to filter.

3A is a flowchart illustrating a method of extracting a QA dataset from a paragraph in the Q&A learning model 100 of FIG. 1A and learning the QA model.

In step 310, the Q&A learning model may sample latent variables from arbitrary paragraphs. In order to sample the latent variable, the Q&A learning model may process contexts included in a paragraph and express it in a vector form.

Thereafter, the Q&A learning model can generate a QA pair, that is, a data set of Q&A, based on the latent variables extracted from the paragraph. (Step 320) The generated QA pair may be used as training data for updating a QA model that performs a function of generating a response to an arbitrary query.

Meanwhile, the question-and-answer learning model may determine whether to apply the generated query and response data, ie, a QA pair, to the learning of the QA model. (Step 330)

Thereafter, the QA model may be updated by learning the QA pair. (Step 340)

Thereafter, the performance of the updated QA model may be measured (step 350), and the measured performance may be applied to steps 310 and 330 as a reward. (Step 360) Therefore, the first agent model that performs the sampling of the latent variable will be updated in the direction that the performance of the QA model increases, and the second agent model that determines whether to apply the QA pair to the QA model is also the performance of the QA model It will be updated in this upward direction.

On the other hand, although not separately shown in FIG. 3A , in step 360, the performance of the QA model measured in step 350 is saturated despite the continuous learning of the QA pair, so that the change range is insufficient, or the performance of the QA model is in advance It may not be performed if it rises above the set range. The former is because learning of the QA pair is no longer conducive to improving the performance of the QA model, and the latter is because the performance of the QA model is sufficient and there is no need to learn the QA pair.

Meanwhile, FIG. 1B is a conceptual diagram schematically illustrating a question-and-answer learning model through adversarial learning according to an additional embodiment of the present invention.

Referring to FIG. 1B , a question-and-answer learning model using adversarial learning according to an embodiment may include an encoder 120 , a response decoder 140 , a query decoder 150 , and a question-and-answer learning model 170 . .

First, in the Q&A learning model, when the paragraph 101 is input, the encoder 120 generates the latent variable 103 with reference to the constraint 122 of the preprocessing 121 step.

When the generated latent variable 103 is input to the response decoder 140 , the response decoder 140 generates a response based on the latent variable 103 . Next, when the generated response is input to the query decoder 150 , the query decoder 150 generates a pair of queries.

In addition, the question-and-answer learning model 170 learns the question-and-answer learning model by inputting the query and response pairs generated by the response decoder 140 and the query decoder 150 .

However, according to an embodiment, it is automatically generated through the question-and-answer dataset generation model generated by the response decoder 140 and the question-and-answer decoder 150, and most of it is of great help because the question-and-answer learning model can already match it. It may not be possible, so a method to improve machine learning performance is needed.

Accordingly, a method of generating a question-and-answer dataset that the current question-and-answer learning model cannot fit through adversarial learning is disclosed.

More specifically, FIG. 2 is a diagram for explaining a method for improving the performance of a question-and-answer model through adversarial learning according to an exemplary embodiment.

Referring to FIG. 2 , in order to perform adversarial learning, it is necessary to sample data in a data manifold while outside the decision boundary of the current question-and-answer model. Specifically, in the method for generating a question-and-answer dataset based on a paragraph input according to an embodiment, a latent variable is selected through an encoder. it should be If so, the Q&A learning model corrects the decision boundary by learning the question-and-answer pair that did not give the correct answer. If this process is repeated, the decision boundary of the Q&A learning model approaches the true decision boundary Performance can also be improved.

On the other hand, the Q&A dataset for such adversarial learning can be determined by adjusting the loss of the Q&A learning model and the constraints to be located in the data manifold.

Referring back to FIG. 1B , the above-described adversarial question-and-answer learning model may have the objective function of Equation 3 below.

는 질의응답 학습모델의 전처리(121),

는 인코더(120),

는 각각 응답디코더(140), 질의디코더(150)에 이용되는 함수이다. 또한 질의응답학습모델(170)로는 Conditional Variational AutoEncoder (CVAE)가 이용될 수 있다.Here, x, y, c, and z correspond to a question, a response, a passage, and a latent variable, respectively. If Equation 3 above is applied to FIG. 1B,

is the preprocessing 121 of the question-and-answer learning model,

is the encoder 120,

is a function used in the response decoder 140 and the query decoder 150, respectively. In addition, a Conditional Variational AutoEncoder (CVAE) may be used as the question-and-answer learning model 170 .

를 샘플링 하는데, 이로부터 생성된 질문과 정답이 질의응답모델의 로스(

)를 높여야 한다. 그러나 이렇게 생성된 질문과 정답 사이에는 관련성이 전혀 없을 수 있기 때문에

를 선택하는 데 있어서 제약조건이 필요하다. 즉, 이전의 학습된

의 분포에서 멀어질수록 유효하지 않은 질문 정답 쌍이 생성되기 때문에, 이를 막기 위해, 두 분포 사이의 거리가 멀어지면 패널티(penalty)를 부여하는 방식으로 값을 조정할 수 있다. (예컨대: KL divergence를 이용).Encoder 120 is a latent variable

is sampled, and the questions and answers generated from it are the loss (

) should be increased. However, since there may not be any relationship between the generated question and the correct answer,

Constraints are required to select That is, previously learned

Since an invalid question answer pair is generated as the distance increases from the distribution of , to prevent this, the value can be adjusted in a manner that imposes a penalty when the distance between the two distributions increases. (eg: using KL divergence).

Accordingly, the question-and-answer learning model according to an embodiment generates a latent variable that is effective but the question-and-answer learning model does not answer the correct answer by adjusting the loss 123 and constraints 122 .

And, by performing adversarial machine learning using the adversarial query and response data set generated using the generated latent variable, the performance of the question-and-answer learning model can be improved.

)를 높여야 한다. 그러나 이렇게 생성된 질문과 정답 사이에는 관련성이 전혀 없을 수 있기 때문에 잠재변수를 선택하는 데 있어서 제약조건이 필요하다. 즉, 이전의 학습된 잠재변수의 분포에서 멀어질수록 유효하지 않은 질문 정답 쌍이 생성되기 때문에, 이를 막기 위해, 두 분포 사이의 거리가 멀어지면 패널티(penalty)를 부여하는 방식으로 값을 조정할 수 있다.Referring to FIG. 3B , in the method of generating a question-and-answer learning model through adversarial learning, in step S300, a latent variable is sampled based on a constraint in an input paragraph. Here, the constraint may be controlled so that the latent variable exists in the data manifold while increasing the loss of the Q&A learning model. For example, to learn the generated question and the correct answer adversarially of the question-and-answer model, the loss (

) should be increased. However, since there may be no relationship between the generated question and the correct answer, constraints are required in selecting the latent variable. In other words, as the distance from the distribution of the previously learned latent variable increases, an invalid question answer pair is generated. .

Next, in step S310, a response is generated based on the latent variable, and a query is generated based on the response in step S320. In step S330, machine learning of the question-and-answer learning model is performed using the query and response data set generated by repeating steps S310 and S320.

Meanwhile, a method of generating a Q&A dataset according to an embodiment will be described in more detail below with reference to FIGS. 4 to 7 .

4 is a conceptual diagram schematically illustrating a question-and-answer dataset generation model based on a paragraph input according to an embodiment.

Referring to FIG. 4 , the question and answer dataset generation model according to an embodiment may include a memory 110 , an encoder 120 , an attention 130 , a response decoder 140 , and a query decoder 150 . .

According to the question-and-answer dataset generating method according to an embodiment, a passage 101 including several sentences is directly input to the encoder 120 . Here, the paragraph 101 may include a phrase, a passage, a paragraph, and the like.

The encoder 120 generates a hidden representation for each word in the input paragraph 101 and stores it in the memory 110 . Here, the encoder 120 may construct a hidden expression from the input through a neural network. Therefore, the hidden representation may mean a machine-readable data representation learned from the hidden layer of the neural network, and may consist of two pairs of a key and a value. For example, the encoder 120 may store information about the meaning indicated by a component (eg, a word) of a sentence as a hidden expression.

Next, the encoder 120 performs sampling 102 based on the hidden expression to generate a latent variable 103 , and the generated latent variable 103 is input to the response decoder 140 .

The response decoder 140 generates a weighted sum vector of the memory 110 by calculating the importance of each component of the memory 110 through the attention 130 using the latent variable 103 as a query. do. For example, at every time step of predicting an output word from the response decoder 140 through the attention 130 , the entire input sentence from the encoder 120 may be referred to once again. However, not all input sentences are referenced at the same rate, but rather the part of the input word related to the word to be predicted at that point in time is referred to with more attention. Therefore, the attention function calculates the degree of similarity with the 'Key' obtained from all hidden expressions for a given 'Query'. And the obtained similarity is reflected in each 'Value' mapped with the key. Then, the weighted sum vector can be obtained by adding all the 'values' reflecting the similarity.

The response decoder 140 predicts an answer span 104 based on the weighted sum vector. Here, the response span may mean the starting point and the ending point where the response is located in the input paragraph.

Next, the query decoder 150 may receive the previously generated response span 104 , calculate the attention 130 of the memory 110 , and finally generate the query 105 .

According to an embodiment, the Q&A dataset including the generated pair of Q&A may be stored in the memory 110, and the stored Q&A data set may be used for machine learning the Q&A learning model.

Therefore, since the dataset generation model according to an embodiment automatically creates a dataset of questions and responses from an inputted paragraph, it is possible to sample several pairs of question-and-answer pairs with only one paragraph input. It can build more diverse data than manual input model.

More specifically, the above-described method for generating a question-and-answer dataset may be implemented through the objective function shown in Equation 4 below.

는 질의응답 학습모델의 인코더,

는 각각 응답디코더, 질의디코더에 이용되는 함수이다.

는 잠재변수의 사전 분포(marginal prior distribution)이며,

는 잠재변수의 사후 분포(marginal posterior distribution)이다. 예를 들면,

는 입력으로부터 생성한 잠재변수의 분포를 생성하고,

는 잠재변수를 이용하여 입력에 가까운 출력을 생성할 수 있다. 한편, 최적화를 위한(상술한 두 분포의 차이를 줄이기 위한) KL(Kullback-Leibler) divergence를 구하는 것은 매우 어려우므로 다음의 수학식 5과 같이 근사할 수도 있다.Here, x, y, c, and z correspond to a question, a response, a passage, and a latent variable, respectively.

is the encoder of the question-and-answer learning model,

is a function used in response decoder and query decoder, respectively.

is the marginal prior distribution of the latent variable,

is the marginal posterior distribution of the latent variable. For example,

creates a distribution of latent variables generated from the input,

can generate an output close to the input using the latent variable. Meanwhile, since it is very difficult to obtain a Kullback-Leibler (KL) divergence for optimization (for reducing the difference between the two distributions described above), it may be approximated as in Equation 5 below.

Meanwhile, how the paragraph input is made and how the question and answer dataset construction model according to an embodiment can be utilized will be described with reference to FIGS. 5 to 7 .

5 is a diagram for explaining an example of a method of generating a question and answer dataset in a paragraph according to an embodiment.

Referring to FIG. 5 , when a short 301 is input according to an embodiment, a response 303 is first generated through sampling 302 . Then, a query 304 paired with the response may be generated using the generated response 303 .

For example, in the input paragraph, "April 28, 1545 ~ December 16, 1598 (November 19 in the lunar calendar) of Yi Sun-sin is a warrior in the mid-Joseon period." , you can first generate the response "April 28, 1545" by sampling. And, based on the response generated according to the method of generating a question-and-answer dataset according to an embodiment, various types of queries such as “the birthday of Chungmugong Yi Sun-sin” or “When was Yi Sun-sin born?” can be created. . Therefore, by using the Q&A generation model, it is possible to construct various data sets for machine learning the Q&A model.

6 is a diagram for explaining a method of generating a question and answer data set in a standardized infobox according to an embodiment.

Referring to FIG. 6 , the question-and-answer dataset generation model according to an embodiment may acquire infobox data 400 through crawling at a site famous for an online knowledge encyclopedia.

In particular, since the infobox data 400 is standardized, it can be easily structured in the form of a corpus of questions and answers, and since these knowledge encyclopedia sites provide detailed classification data for a field or object, a desired field is targeted to generate a question and answer, and an input paragraph can be created in the preprocessing step for machine learning.

However, since not all websites provide the standardized infobox data 400 as shown in FIG. 6 , in general, crawled data is unstructured. Therefore, in this case, let's look at how to create a paragraph to be input.

7 is a diagram for explaining a method of generating a question-and-answer data set from an unstructured page according to an embodiment.

Referring to FIG. 7 , in step S500, the question-and-answer dataset construction model recognizes the title and body of a web page from crawled data. Then, in step S510, the table is recognized based on column recognition in the body. That is, the table is recognized to obtain data in a form similar to that of an infobox that can be obtained from a standardized site.

Next, in step S520, it is checked whether there is a word satisfying a preset data frequency, and the paragraph is formalized. For example, words that appear repeatedly, such as "country name" and "capital", can be identified, and a paragraph can be standardized based on this.

In step S530, the question-and-answer dataset generation model according to an embodiment receives a standardized paragraph and generates a question-and-answer data set, thereby providing data provided in the form of an information box or table provided in a standardized form from an existing website. A dataset of various queries and responses can be built from the set.

The above-described question-and-answer dataset generation model according to another embodiment does not simply generate a question-and-answer dataset, but extracts a question and answer dataset from data that has been trained or machine translated, and increases the accuracy of an answer corresponding to a query. It can be evaluated and used to verify the quality of learning or translation through the evaluation results.

For example, if the question-and-answer dataset generation model according to an embodiment can generate an accurate question-and-answer dataset from machine translation-completed data, it can be evaluated that machine reading is going well.

8 is a block diagram schematically illustrating an internal configuration of an apparatus for generating a question-and-answer learning model according to an embodiment.

Referring to FIG. 8 , the apparatus 100 for generating a question-and-answer learning model according to an embodiment may include one or more memories 110 and a processor 190 . The operation of the Q&A dataset generating apparatus 100 may be performed by executing a program stored in the memory 110 through the processor 190 .

In the operation performed by the processor 190 according to an embodiment, a hidden representation for each word in an input paragraph is generated and stored in the memory 110 .

In addition, the processor 190 samples a latent variable based on the hidden expression, and calculates and weights the importance of the hidden expression stored in the memory through attention using the sampled latent variable as a query. create a vector

In addition, the processor 190 generates a response span based on the latent variable and the weighted sum vector, and generates a query based on the generated response span and attention to the hidden expression of the memory. can

In addition, the processor 190 may further perform an operation of storing the generated query and response data set in the memory 110 and an operation of machine learning the question and answer learning model using the stored query and response data set. have.

In addition, the processor 19 controls the loss and constraints of the question-and-answer learning model to generate a latent variable that is effective but the question-and-answer learning model does not answer the correct answer, and the generated latent variable. An operation for performing adversarial machine learning may be further performed using the generated Q&A dataset.

Meanwhile, the above-described data set generating apparatus 100 may include one or more processors 190 and/or one or more memories 110 . In addition, the processor 190 may include an encoder 120 , a response decoder 140 , and a query decoder 150 , each of the encoder 120 , the response decoder 150 and the query decoder 150 in FIG. 1 . The operation described above with reference to FIG. 7 may be performed.

Also, the memory 110 may include volatile and/or non-volatile memory. The one or more memories 110 may store instructions that, when executed by the one or more processors 190 , cause the one or more processors 190 to perform an operation. In the present disclosure, programs or commands are software stored in the memory 110 , and provide various functions to applications so that an operating system, an application, and/or an application for controlling the resources of the server 100 can utilize the resources of the device. middleware and the like.

The one or more processors 190 may control at least one component of the device 100 connected to the processor 190 by driving software (eg, a program or a command). In addition, the processor 190 may perform various operations, processing, data generation, processing, etc. related to the present disclosure. In addition, the processor 190 may load data or the like from the memory 110 or store it in the memory 110 .

In an embodiment, at least one of the components of the device 100 may be omitted or another component may be added. Also, additionally or alternatively, some components may be integrated and implemented, or may be implemented as a singular or a plurality of entities.

On the other hand, the steps of the method or algorithm described in relation to the embodiment of the present invention may be implemented directly as hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-extractable recording medium well known in the art to which the present invention pertains.

Above, although embodiments of the present invention have been described with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains know that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: Q&A learning model generating device
110: memory
120: encoder
130: attention
140: response decoder
150: query decoder

Claims (11)

A method of operating a question-and-answer model through reinforcement learning in an apparatus for generating a response to a question, the method comprising:
at the first agent, sampling a latent variable in an arbitrary paragraph;
extracting a data set of questions and answers from the paragraphs based on the latent variables;
determining, in the second agent, whether to apply the extracted data set of questions and responses to training of a question-and-answer model for generating a response to an arbitrary query; and
Applying the performance change value of the question and answer model to the first agent and the second agent as a reward,
The sampling of the latent variable may include: generating a hidden representation for each word in the paragraph and storing it in a memory; and sampling the latent variable based on the hidden expression,
The hidden expression includes information indicating the meaning of the word and includes a key used to calculate the importance of the hidden expression and a value associated with the key,
and the second agent is configured to filter at least a portion of the extracted query and response data set to improve performance of the question and answer model based on the reward.
According to claim 1,
The first agent is
Based on the reward, the method characterized in that it is a machine learning model that forms a latent variable space and learns to generate the latent variable from the context included in the paragraph.
The method of claim 2, wherein the extracting of the data set comprises:
generating a weighted sum vector by calculating the importance of the hidden expression stored in the memory through attention using the sampled latent variable as a query; and
Further comprising the step of generating a response span (span) based on the latent variable and the weighted sum vector,
The step of generating the weighted sum vector comprises:
obtaining the key from the hidden expression;
calculating a similarity between the query and the key;
assigning the calculated similarity to the value associated with the key; and
calculating the weighted sum vector based on the value included in the hidden expression.
A method of operating a question-and-answer model through adversarial learning in an apparatus for generating a response to a question, the method comprising:
sampling a latent variable based on a constraint in an arbitrary paragraph;
generating a response based on the latent variable;
generating a query based on the response; and
and machine learning the question and answer learning model using the generated question and answer data set,
The step of sampling the latent variable is
generating a hidden representation for each word in the paragraph and storing it in a memory; and
Further comprising the step of sampling the latent variable based on the hidden expression,
The hidden expression includes information indicating the meaning of the word and includes a key used to calculate the importance of the hidden expression and a value associated with the key,
The constraint is, by giving a penalty when the latent variable is distributed at a location more than a predetermined distance from the distribution of the previously learned latent variable, the latent variable is the loss of the question-and-answer learning model is the height while being controlled to exist within a data manifold.
5. The method of claim 4,
The step of generating the response comprises:
generating a weighted sum vector by calculating the importance of the hidden expression stored in the memory through attention using the sampled latent variable as a query; and
Further comprising the step of generating a response span (span) based on the latent variable and the weighted sum vector,
The step of generating the weighted sum vector comprises:
obtaining the key from the hidden expression;
calculating a similarity between the query and the key;
assigning the calculated similarity to the value associated with the key; and
calculating the weighted sum vector based on the value included in the hidden expression.
5. The method of claim 1 or 4,
The method of claim 1, wherein the paragraph is a structure of the infobox data extracted from the crawled data on the website.
7. The method of claim 6,
When the infobox data is not extracted from the crawled data, a standardized paragraph based on the data frequency in the table recognized through column recognition is input as the paragraph.
8. The method of claim 7,
The above paragraph is data that has been machine translated,
The method further comprising the step of evaluating the quality of the machine translation by evaluating the accuracy of the answer based on the generated query and response data set
one or more processors; and
Comprising one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation,
The one or more processors,
Based on a constraint in an arbitrary paragraph, a latent variable is sampled, a response is generated based on the latent variable, a query is generated based on the response, and a data set of the generated query and response It is configured to machine learning the question-and-answer learning model using
The one or more processors,
A hidden representation for each word in the paragraph is generated and stored in a memory, and the hidden representation-the hidden representation includes information indicating the meaning of the word and is used to calculate the importance of the hidden representation and sample the latent variable based on - including a key and a value associated with the key;
The constraint is, by giving a penalty when the latent variable is distributed at a location more than a predetermined distance from the distribution of the previously learned latent variable, the latent variable is the loss of the question-and-answer learning model is the height while being controlled to exist within a data manifold,
A device for generating a question-and-answer learning model.
Combined with a computer that is hardware, and stored in a medium to execute the method of any one of claims 1 to 5, a question-and-answer learning model generation program through adversarial learning. delete

Images