JP7468654B2 - Question answering device, question answering method, and question answering program - Google Patents

Description

The present disclosure relates to a question answering device, a question answering method, and a question answering program.

Machine reading comprehension is known as a question-answering technology that automatically responds to questions entered by a user in natural language. Machine reading comprehension is a technology that inputs a user's question and a related document written in natural language (called a "passage"), and outputs a response to the input question based on information extracted from the passage.

The output format of the response to the question by the machine reading comprehension is various. For example,
- A format that outputs answer sentences generated by sentence generation based on information extracted from the passage,
- A format that outputs labels such as YES/NO based on information extracted from the passage,
- A format that outputs questions (questions to narrow down answers) generated based on information extracted from the passage,
etc.

Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, and Junji Tomita, "Multi-style generative reading comprehension", In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019), pp. 2273-2284, 2019. Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Itsumi Saito, Atushi Otuka, Hisako Asano and Junji Tomita, "Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019)， pp.2273- 2284．2019． Atsushi Otsuka, Kyosuke Nishida, Itsumi Saito, Hisako Asano, Junji Tomita, and Tetsuji Sato. Proposal of a question answering method using machine comprehension focusing on clarifying the intent of the question. Transactions of the Japanese Society for Artificial Intelligence, vol.34, no.5 A-J14, pp.1-12, 2019.

Here, in order to achieve machine reading comprehension that can output in any of the above output formats, it is possible to consider combining machine reading comprehension models that correspond to each output format.

However, if a number of machine reading comprehension models corresponding to the number of output formats were to be deployed in memory and executed, a large amount of resources would be consumed within the device, and this is unlikely to be feasible on devices with limited resources.

The present disclosure aims to provide a question answering device, a question answering method, and a question answering program that are capable of outputting responses to questions in multiple output formats through machine reading comprehension.

According to one aspect of the present disclosure, a question answering device includes:
a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
A plurality of output units each configured by an output layer of each of the plurality of machine-learned models ,
The multiple output units each output a response in a corresponding output format when the output of the calculation unit is input to the calculation unit in response to a question and a related document used in responding to the question .

According to the present disclosure, it is possible to provide a question answering device, a question answering method, and a question answering program that are capable of outputting responses to questions in multiple output formats through machine reading comprehension.

FIG. 1 is a diagram illustrating an example of a hardware configuration of a question answering device. FIG. 2 is a diagram illustrating a functional configuration of a question answering unit of the comparative example. FIG. 3 is a diagram illustrating an example of a functional configuration of the question answering unit of the question answering device according to the first embodiment. FIG. 4 is a first diagram illustrating an example of an operation in the learning phase of the question answering device according to the first embodiment. FIG. 5 is a second diagram illustrating an example of the operation in the learning phase of the question answering device according to the first embodiment. FIG. 6 is a third diagram illustrating an example of the operation in the learning phase of the question answering device according to the first embodiment. FIG. 7 is a flowchart showing the flow of a question answering process by the question answering device according to the first embodiment. FIG. 8 is a diagram illustrating an example of a functional configuration of the question answering device according to the second embodiment. FIG. 9 is a first diagram illustrating an example of an operation in the learning phase of the question answering device according to the second embodiment. FIG. 10 is a second diagram illustrating an example of the operation in the learning phase of the question answering device according to the second embodiment. FIG. 11 is a third diagram illustrating an example of the operation in the learning phase of the question answering device according to the second embodiment. FIG. 12 is a flowchart showing the flow of a question answering process by the question answering device according to the second embodiment.

Each embodiment will be described below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals, and duplicate descriptions will be omitted.

[First embodiment]
<Hardware Configuration of Question Answering Device>
First, a description will be given of a hardware configuration of a question answering device according to the first embodiment. Fig. 1 is a diagram illustrating an example of a hardware configuration of a question answering device.

As shown in Fig. 1, the question answering device 100 has a processor 101, a memory 102, an auxiliary storage device 103, an I/F (Interface) device 104, a communication device 105, and a drive device 106. Each piece of hardware in the question answering device 100 is connected to each other via a bus 107.

The processor 101 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 101 reads various programs (not shown) installed in the auxiliary storage device 103 onto the memory 102 and executes them.

The memory 102 has a primary storage device such as a Read Only Memory (ROM) or a Random Access Memory (RAM). The processor 101 and the memory 102 form what is known as a computer, and the processor 101 executes various programs read onto the memory 102, causing the computer to realize various functions.

For example, in this embodiment, a computer formed by the processor 101 and the memory 102 realizes the question answering unit 110 by the processor 101 executing a question answering program read onto the memory 102. As will be described later, the question answering unit 110 is configured to reduce consumption of computer resources within the question answering device 100.

The auxiliary storage device 103 stores various programs and various data used when the various programs are executed by the processor 101. For example, in this embodiment, the auxiliary storage device 103 has a learning dataset storage unit 120 and a passage storage unit 130, and stores various data (learning datasets and passages (related documents written in natural language)).

The I/F device 104 is a connection device that connects the input device 140 and the output device 141 to the question answering device 100. The I/F device 104 accepts questions for the question answering device 100 via the input device 140. The I/F device 104 also outputs a response generated by the question answering device 100 to the input question via the output device 141. The input device 140 here includes a device that converts an input question into voice data, a device that converts it into text data, etc. Similarly, the output device 141 here includes a device that outputs a response using voice data, a device that outputs a response using text data, etc.

The communication device 105 is a communication device for communicating with other devices via a network.

The drive device 106 is a device for setting the recording medium 142. The recording medium 142 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 142 may also include semiconductor memories that record information electrically, such as ROMs and flash memories.

The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed recording medium 142 in the drive device 106 and reading out the various programs recorded on the recording medium 142 by the drive device 106. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by downloading them from a network via the communication device 105.

Similarly, the various data stored in each storage unit of the auxiliary storage device 103 is stored, for example, by setting the distributed recording medium 142 in the drive device 106 and reading out the various data recorded on the recording medium 142 by the drive device 106. Alternatively, the various data stored in each storage unit of the auxiliary storage device 103 may be stored by being downloaded from a network via the communication device 105.

<Functional configuration of the question answering unit>
Next, a detailed description will be given of the functional configuration of the question answering unit 110. In order to clarify the characteristics of the functional configuration of the question answering unit 110, a comparative example will first be described in which the functional configuration of a question answering unit is constructed by combining a number of machine reading comprehension models corresponding to the number of output formats.

(1) Functional configuration of the question answering unit of the comparative example Fig. 2 is a diagram showing the functional configuration of the question answering unit of the comparative example. The question answering unit 200 of the comparative example has the following configuration to enable outputting responses to questions through machine reading comprehension in multiple output formats:
Input unit 210,
- Multiple machine reading comprehension models (in the example of FIG. 2, the first machine reading comprehension model 221 to the third machine reading comprehension model 223);
Selection unit 230,
has.

The input unit 210 inputs the input question and a passage (a related document written in a natural language) into each of multiple machine reading comprehension models.

The first machine reading comprehension model 221 outputs an answer sentence generated by sentence generation based on information extracted from the passage as a first response.

The second machine reading comprehension model 222 outputs a label, such as YES/NO, as a second response generated based on information extracted from the passage.

The third machine reading comprehension model 223 outputs a question generated based on information extracted from the passage (a question generated to narrow down the answer (called a revised question)) as a third response.

The selection unit 230 selects and outputs a predetermined number of responses from among the responses output from each of the first machine reading comprehension model 221 to the third machine reading comprehension model 223.

As shown in the question answering unit 200 of the comparative example, by combining a number of machine reading comprehension models corresponding to the number of output formats, it is possible to output responses to questions in a plurality of output formats. On the other hand, the question answering unit 200 of the comparative example has the following problems.
Since a number of machine reading comprehension models corresponding to the number of output formats are expanded in memory and executed, a large amount of computer resources are consumed in the question answering apparatus 100.
It is not possible to select appropriate responses when selecting a predetermined number of responses from among the responses output from each of the first machine reading comprehension model 221 to the third machine reading comprehension model 223. This is because it is not possible to determine which responses are better by simply comparing the first to third responses, and it is necessary to calculate some kind of selection index.

In response to this, the question answering unit 110 of the question answering device 100 according to the first embodiment has a configuration that solves these problems. This is explained in detail below.

3 is a diagram showing an example of a functional configuration of a question answering unit of the question answering device according to the first embodiment. The question answering unit 110 has a configuration for outputting a response to a question by machine reading comprehension in a plurality of output formats while suppressing consumption of computer resources and enabling an appropriate response to a question to be selected from responses in a plurality of output formats.
Input unit 310,
- an understanding layer 320 acting as an input layer;
A first output layer 321, a second output layer 322, and a third output layer 323 functioning as output layers;
- an output decision layer 324, which functions as an output layer;
Selection unit 330,
has.

Of these, the input unit 310 inputs the input question and a passage (a related document written in natural language) to the understanding layer 320.

The understanding layer 320 is an example of a calculation unit that receives a question and a passage as input, calculates information indicating the relevance between the question and the passage on a deep learning vector, and outputs a state vector or a state tensor. The understanding layer 320 can adopt any structure as long as it can receive a question and a passage as input and calculate information indicating the relevance between the question and the passage.

For example, the understanding layer 320 can employ BiDAF (see Non-Patent Document 1) using RNN() or Transformer-based BERT (see Non-Patent Document 2). However, the understanding layer 320 must have a structure that outputs in accordance with the format of the state vector or state tensor that is the input to the output layers subsequent to the understanding layer 320 (first output layer 321 to third output layer 323, output judgment layer 324).

The first output layer 321 to the third output layer 323 are examples of output sections, which take as input information (state vector or state tensor) indicating the relationship between the question and the passage output from the comprehension layer 320, and output a response (the output result of machine reading comprehension).

In the case of FIG. 3, the first output layer 321 outputs an answer sentence generated by sentence generation as a first response. The second output layer 322 outputs a label such as YES/NO as a second response. Furthermore, the third output layer 323 outputs a revised question to narrow down the answers to the input question as a third response.

Note that the first output layer 321 to the third output layer 323 may have any deep learning structure when outputting the first response to the third response, respectively. In addition, the output format output by the first output layer 321 to the third output layer 323 is not limited to an answer sentence, a label, and a revised question, and responses in other output formats may be output.

In the example of Figure 3, the first to third output layers are installed, but any number of output layers may be installed, and multiple output layers having the same structure may be installed. For example, two sentence generation decoder structures may be installed, with one output layer trained to generate answer sentences through sentence generation, and the other output layer trained to generate revised questions through sentence generation.

The output judgment layer 324 is an example of an index calculation unit, which calculates the probability distribution of each response output from the first output layer 321 to the third output layer 323.

Specifically, the output judgment layer 324 has a softmax layer with a number of dimensions corresponding to the number of output formats N (in the example of FIG. 3, N=3). The N-dimensional softmax layer receives a state vector or state tensor from the understanding layer 320 and calculates the probability distribution of each response.

This configuration is effective when the number of output formats N is fixed. On the other hand, if it becomes necessary to add an output format, the number of dimensions of the softmax layer cannot be accommodated, and so re-learning processing must be performed on the entire question answering unit 110.

The selection unit 330 selects the top M responses (M is a predetermined number; in the example of Figure 3, M = 1) in the probability distribution calculated by the output judgment layer 324 from the responses output from each of the first output layer 321 to the third output layer 323, and outputs them to the user via the output device 141.

In this way, the question answering unit 110
・Establish a shared understanding layer for multiple output formats (share the input layer),
Separating the input layer (understanding layer) and the output layer (first output layer 321 to third output layer 323, output judgment layer 324) and modularizing each layer,
The selection unit 330 selects the response to be finally output using the probability distribution of each response calculated by the output determination layer 324 installed in the output layer as a selection index.
As a result, the question answering unit 110 can output a response to a question by machine reading comprehension in a plurality of output formats,
・The input layer is shared to reduce the consumption of computer resources, and
- Selecting the appropriate response by calculating the selection index;
This makes it possible.

<Learning method for the question answering section>
Next, a learning method of the question answering unit 110 will be described. In a learning phase in which learning processing is performed on the question answering unit 110, the question answering unit 110 of the question answering apparatus 100 according to this embodiment first installs a comparison/change unit 410 in place of the selection unit 330. Next, the question answering unit 110 of the question answering apparatus 100 according to this embodiment performs learning processing on each layer (the understanding layer 320, the first output layer 321 to the third output layer 323, and the output judgment layer 324) installed in the question answering unit 110.

In this case, the question answering unit 110 of the question answering device 100 uses a learning dataset that is used when performing learning processing on individual machine reading comprehension models (first machine reading comprehension model 221 to third machine reading comprehension model 223).

Specifically, the question answering unit 110 of the question answering device 100 first sets a flag indicating for which output layer the learning dataset is to be used for the learning process.

Next, the question answering unit 110 of the question answering device 100 stores the correct answer data of the output determination layer 324 in the training dataset. For example, the question answering unit 110 of the question answering device 100 stores vector data in which the value of the dimension corresponding to the set flag is "1" and the values of the other dimensions are "0" in the training dataset as the correct answer data of the output determination layer 324.

Next, the question answering unit 110 of the question answering device 100 calculates the learning loss between the response output from the output layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output layer and the understanding layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering unit 110 of the question answering device 100 ignores responses output from output layers other than the output layer corresponding to the set flag.

In addition, the question answering unit 110 of the question answering device 100 calculates a learning loss between the vector data output from the output determination layer and the corresponding correct answer data, and updates the parameters of the output determination layer based on the calculated learning loss.

Figure 4 is a first diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of Figure 4, it is assumed that a flag indicating that the learning dataset 400 is "a learning dataset used in the learning process for the first output layer 321" is set by the question answering unit 110.

As shown in FIG. 4, the learning dataset 400 includes information items such as “input data”, “correct answer data of the first output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
"First output layer correct answer data" stores correct answer data for an answer sentence generated by sentence generation based on information extracted from the corresponding passage.
The “correct data of the output judgment layer” includes the “first dimension” to the “third dimension”, and stores vector data in which the value of the first dimension is “1”, and the values of the second and third dimensions are “0”.

In Figure 4, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 400 to the understanding layer 320, the first output layer 321 to the third output layer 323 output the first response to the third response as the output results of machine reading comprehension. In addition, the output judgment layer 324 outputs vector data with N dimensions (N=3 in the example of Figure 4).

The comparison/change unit 410 calculates a learning loss between the first response (answer sentence) output from the first output layer 321 and the answer sentence stored in the "correct answer data of the first output layer" of the learning dataset 400. The comparison/change unit 410 also updates the parameters of the first output layer 321 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 4) vector data output from the output judgment layer 324 and the first to third dimensional vector data stored in the "correct data of the output judgment layer" of the learning dataset 400. The comparison/change unit 410 also updates the parameters of the output judgment layer 324 based on the calculated learning loss.

On the other hand, Figure 5 is a second diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of Figure 5, it is assumed that a flag indicating that the learning dataset 500 is "a learning dataset used in the learning process for the second output layer 322" is set by the question answering unit 110.

As shown in FIG. 5, the learning dataset 500 includes information items such as “input data”, “correct answer data of the second output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
"Second output layer correct answer data" stores correct answer data of labels such as YES/NO, which are generated based on information extracted from the corresponding passage.
The “correct data of the output judgment layer” includes the “first dimension” to the “third dimension”, and stores vector data in which the value of the second dimension is “1” and the values of the first and third dimensions are “0”.

In Figure 5, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 500 to the understanding layer 320, the first output layer 321 to the third output layer 323 output the first response to the third response as the output results of machine reading comprehension. In addition, the output judgment layer 324 outputs vector data with N dimensions (N=3 in the example of Figure 5).

The comparison/change unit 410 calculates a learning loss between the second response (label) output from the second output layer 322 and the label stored in the "correct answer data of the second output layer" of the learning dataset 500. The comparison/change unit 410 also updates the parameters of the second output layer 322 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 5) vector data output from the output judgment layer 324 and the first to third dimensional vector data stored in the "correct data of the output judgment layer" of the learning dataset 500. The comparison/change unit 410 also updates the parameters of the output judgment layer 324 based on the calculated learning loss.

On the other hand, Figure 6 is a third diagram showing an example of operation in the learning phase of the question answering device according to the first embodiment. In the case of Figure 6, it is assumed that a flag indicating that the learning dataset 600 is "a learning dataset used in the learning process for the third output layer 323" is set by the question answering unit 110.

As shown in FIG. 6, the learning dataset 600 includes information items such as “input data”, “correct answer data of the third output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
The "third output layer correct answer data" stores the correct answer data for the revised question generated based on the information extracted from the corresponding passage.
The “correct answer data of the output judgment layer” includes the “first dimension” to the “third dimension”, and stores vector data in which the value of the third dimension is “1” and the values of the first and second dimensions are “0”.

In Figure 6, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 600 to the understanding layer 320, the first output layer 321 to the third output layer 323 output the first response to the third response as the output results of machine reading comprehension. In addition, the output judgment layer 324 outputs vector data with N dimensions (N=3 in the example of Figure 6).

The comparison/change unit 410 calculates a learning loss between the third response (revised question) output from the third output layer 323 and the revised question stored in the "correct answer data of the third output layer" of the learning dataset 600. The comparison/change unit 410 also updates the parameters of the third output layer 323 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 410 calculates a learning loss between the N-dimensional (N=3 in the example of FIG. 6) vector data output from the output judgment layer 324 and the first to third dimensional vector data stored in the "correct data of the output judgment layer" of the learning dataset 600. The comparison/change unit 410 also updates the parameters of the output judgment layer 324 based on the calculated learning loss.

In this way, the question answering unit 110 of the question answering device 100 uses the learning datasets 400 to 600 to sequentially perform learning processes on the understanding layer 320, the first output layer 321 to the third output layer 323, and the output judgment layer 324.

<Question answering process flow>
Next, a flow of question answering processing by the question answering device 100 will be described. Fig. 7 is a flowchart showing the flow of question answering processing by the question answering device according to the first embodiment. Of these, steps S701 to S703 represent processing in the learning phase, and steps S704 to S707 represent processing in the response phase.

In step S701, the question answering unit 110 uses the learning dataset 400 to perform learning processing on the understanding layer 320, the first output layer 321, and the output judgment layer 324.

In step S702, the question answering unit 110 uses the learning dataset 500 to perform learning processing on the understanding layer 320, the second output layer 322, and the output judgment layer 324.

In step S703, the question answering unit 110 uses the learning dataset 600 to perform learning processing on the understanding layer 320, the third output layer 323, and the output judgment layer 324.

In step S704, the input unit 310 of the question answering unit 110 accepts input of a question and a passage, and inputs the input question and passage to the understanding layer 320.

In step S705, the first output layer 321 to the third output layer 323 use the state vector output from the understanding layer 320 as input and output the first response to the third response.

In step S706, the output judgment layer 324 of the question answering unit 110 uses the state vector output from the understanding layer 320 as input, calculates the probability distribution of the first to third responses, and outputs a selection index.

In step S707, the selection unit 330 of the question answering unit 110 selects the top M predetermined responses based on the selection index output from the output determination layer 324, and outputs the selected responses.

<Summary>
As is apparent from the above description, the question answering apparatus 100 according to the first embodiment has the following features:
-Has an understanding layer that takes a question and a passage as input and calculates information indicating the relevance between the question and the passage.
- It has first to third output layers, which use information indicating the relevance calculated by the understanding layer as their respective inputs and output first to third responses which are mutually different output formats.
The present invention has an output judgment layer that calculates a probability distribution of the first to third responses based on the information indicating the relevance output by the understanding layer, and further has a selection unit that selects a predetermined number of responses using the probability distribution of the first to third responses calculated by the output judgment layer as a selection index.

As a result, the question answering device 100 according to the first embodiment can output responses to questions in multiple output formats, while reducing the consumption of computer resources and enabling the selection of an appropriate response.

In other words, according to the first embodiment, it is possible to provide a highly feasible question answering device, question answering method, and question answering program that are capable of outputting responses to questions through machine reading in multiple output formats.

Second Embodiment
In the above first embodiment, it is assumed that the number of output formats is fixed, and when a new output format is added, the question answering unit is configured to perform a re-learning process for the entire question answering unit 110.

In contrast, in the second embodiment, the question answering unit is configured so that even if a new output format is added, there is no need to perform re-learning processing for the entire question answering unit 110. The second embodiment will be described below, focusing on the differences from the first embodiment.

<Functional configuration of the question answering unit>
First, a description will be given of a functional configuration of a question answering unit of the question answering device according to the second embodiment. Fig. 8 is a diagram illustrating an example of a functional configuration of the question answering unit of the question answering device according to the second embodiment.

The difference with the functional configuration shown in Figure 3 is that in the case of the question answering unit 800 in Figure 8, a first output judgment layer 801 to a third output judgment layer 803 are provided which calculate individual scores as selection indices for the first output layer 321 to the third output layer 323, respectively. Also, in the case of the question answering unit 800 in Figure 8, the function of the selection unit 810 is different from the function of the selection unit 330 in Figure 3.

The first output determination layer 801 is an example of an index calculation unit, and has a logit layer that receives the state vector of the first output layer 321 and calculates a scalar value between 0 and 1.0 as the first score.

Similarly, the second output decision layer 802 is an example of an index calculation unit, and has a logit layer that receives the state vector of the second output layer 322 and calculates a scalar value between 0 and 1.0 as the second score.

Similarly, the third output decision layer 803 is an example of an index calculation unit, and has a logit layer that receives the state vector of the third output layer 323 and calculates a scalar value between 0 and 1.0 as the third score.

The selection unit 810 selects and outputs responses corresponding to the top M predetermined scores based on the first score to the third score calculated by the first output decision layer 801 to the third output decision layer 803.

In this way, the question answering device 100 according to the second embodiment is provided with a first output judgment layer 801 to a third output judgment layer 803 that calculate individual scores for each of the first output layer 321 to the third output layer 323. As a result, according to the second embodiment, even if a new output format is added, it is sufficient to perform learning processing for the added new output layer and output judgment layer and the understanding layer, and there is no need to perform re-learning processing for the output layer and output judgment layer that have already been trained.

<Learning method for the question answering section>
Next, a learning method of the question answering unit 800 will be described. In a learning phase in which learning processing is performed on the question answering unit 110, the question answering unit 800 of the question answering apparatus 100 according to this embodiment first installs a comparison/change unit 910 instead of the selection unit 810. Next, the question answering unit 800 of the question answering apparatus 100 according to this embodiment performs learning processing on each layer (the understanding layer 320, the first output layer 321 to the third output layer 323, and the first output determination layer 801 to the third output determination layer 803) installed in the question answering unit 800.

In this case, as in the first embodiment described above, the question answering device 100 uses a learning dataset that is used when performing learning processing on individual machine reading comprehension models (first machine reading comprehension model 221 to third machine reading comprehension model 223).

Specifically, the question answering unit 800 of the question answering device 100 first sets a flag indicating for which output layer the learning dataset is to be used for the learning process.

Next, the question answering unit 800 of the question answering device 100 stores the correct answer data of the first score to the third score calculated by the first output judgment layer 801 to the third output judgment layer 803 in the training dataset. For example, the question answering unit 800 of the question answering device 100 stores the correct answer data in which the score corresponding to the set flag is "1.0" in the training dataset.

Next, the question answering unit 800 of the question answering device 100 calculates the learning loss between the response output from the output layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output layer and the understanding layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering unit 800 of the question answering device 100 ignores the responses output from output layers other than the output layer corresponding to the set flag.

In addition, the question answering unit 800 of the question answering device 100 calculates the learning loss between the score output from the output determination layer corresponding to the set flag and the corresponding correct answer data, and updates the parameters of the output determination layer corresponding to the set flag based on the calculated learning loss. At this time, the question answering device 100 ignores scores output from output determination layers other than the output determination layer corresponding to the set flag.

Figure 9 is the first diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of Figure 9, it is assumed that a flag indicating that the learning dataset 900 is "a learning dataset used in the learning process for the first output layer 321 and the first output judgment layer 801" is set by the question answering unit 800.

As shown in FIG. 9, the learning dataset 900 includes information items such as “input data”, “correct answer data of the first output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
"First output layer correct answer data" stores correct answer data for an answer sentence generated by sentence generation based on information extracted from the corresponding passage.
In the "correct data of the output determination layer", the correct data of the score (first score) output from the first output determination layer 801 is stored.

In Figure 9, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 900 to the comprehension layer 320, the first output layer 321 to the third output layer 323 output a first response to a third response as the output result of machine reading comprehension. Also, the first output judgment layer 801 to the third output judgment layer 803 output a first score to a third score.

The comparison/change unit 910 calculates a learning loss between the first response (answer sentence) output from the first output layer 321 and the answer sentence stored in the "correct answer data of the first output layer" of the learning dataset 900. The comparison/change unit 910 also updates the parameters of the first output layer 321 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 910 calculates a learning loss between the first score output from the first output judgment layer 801 and the value stored in the first score of the "correct data of the output judgment layer" in the learning dataset 900. The comparison/change unit 910 also updates the parameters of the first output judgment layer 801 based on the calculated learning loss.

On the other hand, Figure 10 is a second diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of Figure 10, it is assumed that a flag indicating that the learning dataset 1000 is "a learning dataset used in the learning process for the second output layer 322 and the second output judgment layer 802" is set by the question answering unit 800.

As shown in FIG. 10, the learning dataset 1000 includes information items such as “input data”, “correct answer data of the second output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
"Second output layer correct answer data" stores correct answer data of labels such as YES/NO, which are generated based on information extracted from the corresponding passage.

The "correct data of the output determination layer" stores the correct data of the score (second score) output from the second output determination layer 802.

In Figure 10, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 1000 to the comprehension layer 320, the first output layer 321 to the third output layer 323 output a first response to a third response as the output result of machine reading comprehension. In addition, the first output judgment layer 801 to the third output judgment layer 803 output a first score to a third score.

The comparison/change unit 910 calculates a learning loss between the second response (label) output from the second output layer 322 and the label stored in the "correct data of the second output layer" of the learning dataset 1000. The comparison/change unit 910 also updates the parameters of the second output layer 322 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 910 calculates a learning loss between the second score output from the second output judgment layer 802 and the value stored in the second score of the "correct data of the output judgment layer" in the learning dataset 1000. The comparison/change unit 910 also updates the parameters of the second output judgment layer 802 based on the calculated learning loss.

On the other hand, Figure 11 is a third diagram showing an example of operation in the learning phase of the question answering device according to the second embodiment. In the case of Figure 11, it is assumed that a flag indicating that the learning dataset 1100 is "a learning dataset used in the learning process for the third output layer 323 and the third output judgment layer 803" is set by the question answering unit 800.

As shown in FIG. 11, the learning dataset 1100 includes information items such as “input data”, “correct answer data of the third output layer”, and “correct answer data of the output judgment layer”.
・"Input data" stores questions and passages.
The "third output layer correct answer data" stores the correct answer data for the revised question generated based on the information extracted from the corresponding passage.
In the "correct data of the output determination layer", the correct data of the score (third score) output from the third output determination layer 803 is stored.

In Figure 11, when the input unit 310 inputs input data (pairs of questions and passages) from the learning dataset 1100 to the understanding layer 320, the first output layer 321 to the third output layer 323 output a first response to a third response as the machine learning output results. In addition, the first output judgment layer 801 to the third output judgment layer 803 output a first score to a third score.

The comparison/change unit 910 calculates a learning loss between the third response (revised question) output from the third output layer 323 and the revised question stored in the "correct answer data of the third output layer" of the training dataset 1100. The comparison/change unit 910 also updates the parameters of the third output layer 323 and the understanding layer 320 based on the calculated learning loss.

Similarly, the comparison/change unit 910 calculates a learning loss between the third score output from the third output judgment layer 803 and the value stored in the third score of the "correct data of the output judgment layer" in the learning dataset 1100. The comparison/change unit 910 also updates the parameters of the third output judgment layer 803 based on the calculated learning loss.

<Question answering process flow>
Next, a flow of a question answering process by the question answering device 100 according to the second embodiment will be described. Fig. 12 is a flowchart showing the flow of a question answering process by the question answering device according to the second embodiment. The difference from the flowchart described in Fig. 7 in the first embodiment is steps S1201 to S1203.

In step S1201, the question answering unit 110 uses the learning dataset 900 to perform learning processing on the understanding layer 320, the first output layer 321, and the first output judgment layer 801.

In step S1202, the question answering unit 110 uses the learning dataset 1000 to perform learning processing on the understanding layer 320, the second output layer 322, and the second output judgment layer 802.

In step S1203, the question answering unit 110 uses the learning dataset 1100 to perform learning processing on the understanding layer 320, the third output layer 323, and the third output judgment layer 803.

<Summary>
As is apparent from the above description, the question answering apparatus 100 according to the second embodiment has the following features:
-Has an understanding layer that takes a question and a passage as input and calculates information indicating the relevance between the question and the passage.
- It has first to third output layers, which use information indicating the relevance calculated by the understanding layer as their respective inputs and output first to third responses which are mutually different output formats.
The system has a first output determination layer to a third output determination layer that receive state vectors of the first output layer to the third output layer and calculate individual scores (first score to third score) for the first output layer to the third output layer, and further has a selection unit that selects a predetermined number of responses using the first score to the third score calculated by the first output determination layer to the third output determination layer as selection indexes.

As a result, according to the question answering device 100 of the second embodiment, like the first embodiment, responses to questions can be output in multiple output formats, while reducing the consumption of computer resources and enabling the selection of an appropriate response. In addition, according to the question answering device 100 of the second embodiment, even when a new output format is added, there is no need to perform a re-learning process for the entire question answering unit.

In other words, according to the second embodiment, it is possible to provide a more feasible question answering device, question answering method, and question answering program that can output responses to questions through machine reading comprehension in multiple output formats.

[Other embodiments]
In the above first and second embodiments, the cases where different output determination layers are installed have been described, but the decision as to whether to install the output determination layer in the first embodiment or the output determination layer in the second embodiment is arbitrary. For example, the decision may be made in consideration of the tasks and objectives to be set, the system configuration of the question answering device, and the like.

In addition, in the above first and second embodiments, the learning phase and the response phase are described as being executed in the same question answering device 100. However, the learning phase and the response phase may be configured to be executed in separate devices. In this case, the device that executes the response phase does not need to have a learning dataset storage unit 120, and the comparison/modification unit 410, 910 is not installed.

Note that the present invention is not limited to the configurations described in the above embodiments, and may be combined with other elements. These points can be modified without departing from the spirit of the present invention, and can be appropriately determined according to the application form.

100: Question answering device 110: Question answering unit 120: Learning dataset storage unit 130: Passage storage unit 310: Input unit 320: Understanding layer 321: First output layer 322: Second output layer 323: Third output layer 324: Output judgment layer 330: Selection unit 400-600: Learning dataset 801: First output judgment layer 802: Second output judgment layer 803: Third output judgment layer 810: Selection unit 900-1100: Learning dataset

Claims (9)

a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
A plurality of output units each configured by an output layer of each of the plurality of machine-learned models,
The multiple output units each output a response in a corresponding output format when a question and a related document used in answering the question are input to the calculation unit. a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
A plurality of output units each configured by an output layer of each of the plurality of machine-learned models ;
a selection unit that selects a predetermined number of responses from among the responses in each output format output from each of the multiple output units by inputting an output of the calculation unit when a question and a related document used in answering the question are input to each of the multiple output units . An index calculation unit receives an output of the calculation unit and calculates an index value for the selection unit to select a predetermined number of responses,
The question answering apparatus according to claim 2 , wherein the selection unit selects a predetermined number of responses based on the index value. The index calculation unit receives the output of the calculation unit and calculates a probability distribution of each response output by the multiple output units in each output format;
The question answering apparatus according to claim 3 , wherein the selection unit selects a predetermined number of responses based on a probability distribution of each of the responses. Further comprising a plurality of index calculation units that receive information calculated when the plurality of output units output each response and calculate an index value for each of the plurality of output units,
The question answering apparatus according to claim 2 , wherein the selection unit selects a predetermined number of responses based on index values for each of the plurality of output units. The computer
a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
The plurality of machine-learned models function as a plurality of output units each composed of an output layer;
A question answering method in which the multiple output units each receive an output from the calculation unit when a question and a related document used in answering the question are input to the calculation unit, and output a response in a corresponding output format. The computer
a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which is machine-learned using a question, a related document used in answering the question, and a response to the question in each output format, the machine-learned models each having a plurality of layers;
A plurality of output units each configured by an output layer of each of the plurality of machine-learned models;
A question answering method in which the output of the calculation unit when a question and related documents used in answering the question are input is input to each of the multiple output units, and the calculation unit functions as a selection unit that selects a predetermined number of responses from among the responses in each output format output by each of the multiple output units. Computer,
a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
The machine-learned models are each configured as a plurality of output units each composed of an output layer,
A question answering program, wherein the plurality of output units each output a response in a corresponding output format when a question and a related document used in answering the question are input to the calculation unit. Computer,
a calculation unit configured by a common input layer of a plurality of machine-learned models, the number of which corresponds to the number of output formats, each of which has a plurality of layers, the machine-learned models being machine-learned using a question, a related document used in answering the question, and responses to the question in each output format;
A plurality of output units each configured by an output layer of each of the plurality of machine-learned models;
A question answering program that functions as a selection unit that selects a predetermined number of responses from among the responses in each output format output by each of the multiple output units by inputting the output of the calculation unit when a question and related documents used in answering the question are input to each of the multiple output units .

Images