WO2019235103A1 - Question generation device, question generation method, and program - Google Patents

Question generation device, question generation method, and program Download PDF

Info

Publication number
WO2019235103A1
WO2019235103A1 PCT/JP2019/017805 JP2019017805W WO2019235103A1 WO 2019235103 A1 WO2019235103 A1 WO 2019235103A1 JP 2019017805 W JP2019017805 W JP 2019017805W WO 2019235103 A1 WO2019235103 A1 WO 2019235103A1
Authority
WO
WIPO (PCT)
Prior art keywords
question
revised
answer
generation
question sentence
Prior art date
Application number
PCT/JP2019/017805
Other languages
French (fr)
Japanese (ja)
Inventor
淳史 大塚
京介 西田
いつみ 斉藤
光甫 西田
久子 浅野
準二 富田
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2018214187A external-priority patent/JP7087938B2/en
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to US16/972,187 priority Critical patent/US11972365B2/en
Publication of WO2019235103A1 publication Critical patent/WO2019235103A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/9032—Query formulation
    • G06F16/90332—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/90335—Query processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present invention relates to a question generation device, a question generation method, and a program.
  • Non-Patent Document 1 a machine reading type question answering technique for extracting a part which becomes an answer from a document written in a natural language for a question inputted in a natural language is known (for example, Non-Patent Document 1).
  • Machine reading comprehension type question answering technology uses a neural network to collate the question with the answer part described in a manual or other document, and is known to be able to achieve answer accuracy equivalent to or better than that of humans. ing.
  • the question content in order to achieve high answer accuracy in the machine-reading-type question answering technology, the question content must be clear and the information necessary for answering must be included in the question without any shortage.
  • the question content in an actual service using a machine-reading-type question answering technique, the question content may be ambiguous or the question sentence may be too short. In such a case, there is a possibility that the answer to the question cannot be uniquely determined or the answer content may be wrong, and high answer accuracy may not be achieved.
  • An embodiment of the present invention has been made in view of the above points, and an object thereof is to achieve high accuracy in answering a question.
  • a question generation device receives a question sentence and a related document including an answer to the question sentence as input, and uses a machine learning model that has been learned in advance, And generating means for generating a revised question sentence in which a potentially missing part of the question sentence is supplemented with a word included in a predetermined vocabulary set.
  • the revised question (RQ: Revised Question) of the input question (hereinafter, also simply referred to as “input question”) for the purpose of improving the accuracy of answering the question response using the machine-reading-type question answering technology.
  • the question generation device 100 that generates The revised question is a question sentence with more specific contents that reinforces the question contents of the input question. That is, the revised question is a question in which the content of the question is clear and the information necessary for the answer is included without a shortage.
  • the revised question of the question is generated, and then the question answering task using the revised question is performed to improve the answer accuracy of the question answer. Will be able to.
  • each embodiment described below is only an example, and forms to which the present invention can be applied are not limited to the following embodiments.
  • the technology according to each embodiment of the present invention can be used for, for example, a service that provides an answer to a question input by a user in a natural language, but the usage target is not limited to this and can be used for various targets. is there.
  • the question generation device 100 when an input question and a document related to the input question (hereinafter also referred to as “related document”) are given, the question generation device 100 generates a revised question.
  • the revised question of the input question is generated using a machine learning model (hereinafter also referred to as a “revised question generation model”).
  • a revised question generation model is used to match an input question with a related document, and a potentially missing part of the input question (characters such as words and phrases).
  • a revised question is generated by supplementing the column.
  • the revised question is generated using the related document, so that, for example, a revised question that can be answered by the system that performs the question answering task can be generated (in other words, the system that performs the question answering task is Unanswered revised questions can be prevented from being generated.
  • an input question used as correct answer data a question in which a part of the input question is missing (also referred to as a “missing question”), and a related document are used.
  • Learn the revised question generation model In this learning, the parameters of the revised question generation model are updated so that the natural sentence obtained using the missing question and the related document approaches the input question that is correct answer data.
  • a missing question is a question sentence in which necessary information (a character string such as a word or a phrase) is partially missing as a question sentence related to an input related document.
  • a natural sentence is a sentence written in a natural language.
  • the input question is a sentence written in a natural language (that is, a natural sentence).
  • a natural language that is, a natural sentence.
  • a set of J word tokens Q It is assumed that ⁇ q 0 , q 1 ,..., Q J ⁇ 1 ⁇ .
  • the sentence used as an input question may be, for example, a sentence in which keywords are simply listed in addition to a natural sentence.
  • the sentence etc. which were obtained as a speech recognition result may be sufficient.
  • the related document includes information serving as an answer to the input question. Examples of the related document include a manual in which an answer to an input question is described.
  • the related document is also referred to as a passage.
  • FIG. 1 is a diagram illustrating an example of a functional configuration of a question generating device 100 at the time of generating a revised question in the first embodiment of the present invention.
  • the question generation device 100 for generating a revised question in the first embodiment of the present invention includes a revised question generation unit 200.
  • the revised question generation unit 200 is realized by a learned revised question generation model (that is, a revised question generation model using parameters updated by a revised question generation model learning unit 400 described later).
  • the revised question generation unit 200 inputs a question (input question) and a related document, and generates and outputs a revised question. More specifically, the revised question generation unit 200 generates a revised question by regarding the input question as a missing question and restoring the previous question text using the related document.
  • the revised question generation unit 200 includes a collation unit 210 and a question restoration unit 220.
  • the collation unit 210 generates matching information between the input question and the related document.
  • the matching information is information representing a matching relationship between each word included in the input question and each word included in the related document.
  • the question restoration unit 220 uses the matching information generated by the matching unit 210, the input question, and the related document to generate (restore) a natural sentence so that the input question becomes a question sentence before being lost.
  • the natural sentence generated by the question restoration unit 220 becomes a revised question.
  • FIG. 2 is a diagram illustrating an example of a functional configuration of the question generation device 100 during learning according to the first embodiment of the present invention.
  • the question generation device 100 at the time of learning in the first embodiment of the present invention includes a missing question creation unit 300 and a revised question generation model learning unit 400.
  • the missing question creation unit 300 creates a missing question by inputting a question (input question) and missing a part of the input question.
  • the revised question generation model learning unit 400 learns a revised question generation model using the missing question created by the missing question creation unit 300, the input question, and the related document. Then, the revised question generation model learning unit 400 outputs the learned parameters of the revised question generation model.
  • the revised question generation model learning unit 400 includes a verification unit 210, a question restoration unit 220, and a parameter update unit 410.
  • the collation unit 210 and the question restoration unit 220 are as described above.
  • the parameter update unit 410 calculates an error between the natural sentence (revised question) generated by the question restoration unit 220 and the input question, and uses the error to change the parameter of the revised question generation model by an arbitrary optimization method. (Revised question generation model parameter not learned) is updated.
  • the revised parameter generation model is learned by updating the parameters by the parameter updating unit 410.
  • the revised question generation model is a machine learning model realized by a neural network.
  • all or part of the revised question generation model may be realized by a machine learning model other than the neural network.
  • at least one functional unit of the matching unit 210 and the question restoration unit 220 may be realized by a machine learning model other than the neural network.
  • FIG. 3 is a diagram illustrating an example of a hardware configuration of the question generation device 100 according to the first embodiment of the present invention.
  • the question generation device 100 includes an input device 501, a display device 502, an external I / F 503, a RAM (Random Access Memory) 504, and a ROM (Read Only Memory) 505, an arithmetic device 506, a communication I / F 507, and an auxiliary storage device 508.
  • Each of these hardware is connected via a bus B so as to be able to communicate.
  • the input device 501 is, for example, a keyboard, a mouse, a touch panel, or the like, and is used by a user to input various operations.
  • the display device 502 is a display or the like, for example, and displays a processing result (for example, a revised question or the like) of the question generation device 100.
  • the question generation device 100 may not include at least one of the input device 501 and the display device 502.
  • External I / F 503 is an interface with an external device.
  • the external device includes a recording medium 503a and the like.
  • the question generation device 100 can read and write the recording medium 503a and the like via the external I / F 503.
  • the recording medium 503a may store one or more programs that realize each functional unit included in the question generation device 100.
  • Examples of the recording medium 503a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), a USB (Universal Serial Bus) memory card, and the like.
  • the RAM 504 is a volatile semiconductor memory that temporarily stores programs and data.
  • the ROM 505 is a nonvolatile semiconductor memory that can retain programs and data even when the power is turned off.
  • the ROM 505 stores, for example, settings related to an OS (Operating System), settings related to a communication network, and the like.
  • the computing device 506 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and reads a program or data from the ROM 505, the auxiliary storage device 508, or the like onto the RAM 504 and executes processing.
  • Each functional unit included in the question generation device 100 is realized by, for example, processing that the arithmetic device 506 causes one or more programs stored in the auxiliary storage device 508 to execute.
  • the question generation device 100 may include both the CPU and the GPU as the arithmetic device 506, or may include only one of the CPU and the GPU.
  • the communication I / F 507 is an interface for connecting the question generating device 100 to a communication network.
  • One or more programs that realize each functional unit included in the question generation device 100 may be acquired (downloaded) from a predetermined server device or the like via the communication I / F 507.
  • the auxiliary storage device 508 is, for example, an HDD or SSD (Solid State Drive), and is a non-volatile storage device that stores programs and data.
  • the programs and data stored in the auxiliary storage device 508 include, for example, an OS and one or more programs that realize each functional unit included in the question generation device 100.
  • the question generation device 100 according to the first embodiment of the present invention has the hardware configuration shown in FIG. In the example illustrated in FIG. 3, the case where the question generation device 100 according to the first embodiment of the present invention is realized by one device (computer) is described, but the present invention is not limited thereto.
  • the question generation device 100 according to the first embodiment of the present invention may be realized by a plurality of devices (computers). Further, the question generation device 100 according to the first embodiment of the present invention may be realized by a device (computer) including a plurality of arithmetic devices 506 and a plurality of memories (RAM 504, ROM 505, auxiliary storage device 508, etc.). .
  • FIG. 4 is a flowchart showing an example of a revision question generation process in the first embodiment of the present invention.
  • the revised question generation process it is assumed that the revised question generation model for realizing the revised question generation unit 200 has been learned.
  • FIG. 5 shows an example of a revised question generation model for realizing the revised question generation unit 200 in the first embodiment of the present invention.
  • the revised question generation model is a neural network composed of three layers of Encode ⁇ Layer, Matching ⁇ ⁇ Layer, and Decode Layer.
  • the matching unit 210 is realized by Encode layer and Matching layer.
  • the question restoration unit 220 is realized by Decode layer.
  • the Encode Layer and Decode Layer are layers based on the language generation model Seq2Seq.
  • Matching Layer is a layer based on Attention Flow Layer and Modeling Layer used in machine reading tasks.
  • Seq2Seq refer to Reference Document 1 and Reference Document 2 below, for example.
  • Reference Document 3 For details of the reading comprehension task, refer to Reference Document 3 below, for example.
  • Step S102 Revised Question matching unit 210 of the generator 200, the following steps S102-1 ⁇ step S102-4, as the matching information, a hidden state vector h d0 to the initial state of the Decoder, used in the machine reading task A matching matrix M that is a matching model is generated.
  • Step S102-1 First, the collation unit 210 converts the related document X and the input question Q into d-dimensional word vector sequences as Word Embedding processing in Encode ⁇ Layer of the revised question generation model shown in FIG. That is, the matching unit 210 vectorizes each word token constituting the related document X and the input question Q to create a word vector series.
  • the word vector sequence of the related document X is also represented by X
  • the word vector sequence X of the related document X is
  • the word vector series X and Q are generated from the input question Q and the related document X that are input.
  • the present invention is not limited to this.
  • the word vector series X And Q may be input.
  • Step S102-2 Next, as a process of Passage Context in the Encode Layer of the revised question generation model shown in FIG. 5, the collation unit 210 encodes the word vector sequence X by RNN (Recurrent Neural Network), and the related document X To obtain a context matrix H ⁇ R 2d ⁇ T. Note that a column vector composed of elements of the t-th column of the context matrix H is represented as a context vector H t .
  • RNN Recurrent Neural Network
  • the collation unit 210 encodes the word vector sequence Q by RNN as the Question Context processing in the Encode Layer of the revised question generation model shown in FIG. 5, and sets the context matrix U ⁇ R 2d ⁇ J of the input question Q. obtain.
  • a column vector composed of elements in the j-th column of the context matrix U is represented as a context vector U j .
  • the RNN used for the processing of Passage Context and QuestionQuestContext may be, for example, bi-RNN, LSTM (Long Short Term Memory), bi-LSTM, or the like.
  • LSTM Long Short Term Memory
  • a common parameter is used for the RNN used for the PassagePassContext processing and the RNN used for the Question Context processing.
  • Step S102-3 Next, matching unit 210, as processing of Matching Layer revised question generator model shown in FIG. 5, below, to generate a hidden state vector h d0 to the initial state of the Decoder.
  • the collation unit 210 uses the attention mechanism (attention) to the attention vector U J ⁇ 1 and the context matrix H for the attention vector with the related document X according to the following equations (1) and (2). to calculate the H ⁇ U ⁇ R 2d.
  • attention mechanism attention
  • H ⁇ U ⁇ R 2d For convenience of description, “X with“ ⁇ ”attached to the top (that is, X with“ ⁇ ”added as an accent) is expressed as“ X ⁇ ”.
  • ⁇ represents transposition.
  • Softmax t represents the t-th output of the softmax function. Note that “U” subscripted by H ⁇ U in the above formula (2) is not a subscript.
  • the collation unit 210 uses the attention mechanism (attention) to perform the attention with the input question Q by the following expressions (3) and (4) with respect to the context vector U J ⁇ 1 and the context matrix U. to calculate the vector U ⁇ U ⁇ R 2d.
  • attention mechanism attention
  • softmax j represents the jth output of the softmax function.
  • U subscripted by U ⁇ U in the above formula (4) is not a subscript.
  • the matching unit 210 sets the initial state of the Decoder according to the following equation (5) using the two attention vectors H ⁇ U and U ⁇ U calculated in the above equations (2) and (4), respectively.
  • the hidden state vector hd0 is calculated.
  • W m ⁇ R 4d ⁇ 2d and b m ⁇ R 2d are parameters.
  • F is an activation function, and for example, Leaky ReLU or the like is used.
  • [;] represents a connection.
  • Step S102-4 Next, the collation unit 210 generates a matching matrix M as follows as the Matching layer processing of the revised question generation model shown in FIG.
  • the collation unit 210 inputs a context matrix H having a sequence length of T and a context matrix U having a sequence length of J to the attention layer. And the collation part 210 calculates the similarity matrix S of the word of the related document X and the input question Q as a process of Attention layer.
  • the collation unit 210 calculates attentions in two directions, an attention from the related document X to the input question Q and an attention from the input question Q to the related document X.
  • the collation unit 210 calculates an attention vector weighted by the word of the input question Q for each word of the related document X. That is, the collation unit 210 uses the following equations (7) and (8) to indicate the attention vector corresponding to the t-th word of the related document X.
  • the collation unit 210 calculates an attention vector weighted by a word strongly related to any word of the input question Q, and then uses this attention vector as the related document X.
  • a matrix arranged for the sequence length T is created. That is, first, the matching unit 210 uses the following equations (9) and (10) to obtain an attention vector.
  • the matching unit 210 is a matrix in which T attention vectors calculated by the above equation (10) are arranged.
  • the matching unit 210 uses the attention vector H ⁇ H ⁇ R 2d ⁇ T obtained by taking the self-attention between the context vector H T-1 and the context matrix H, by the following equation (11), and the attention matrix G Calculate
  • the matching unit 210 may calculate the attention matrix G without using the attention vector H ⁇ H ⁇ R 2d (that is, without concatenating the attention vector H ⁇ H in the above equation (11)). .
  • the attention matrix G is G ⁇ R 8d ⁇ T.
  • the matching unit 210 as processing of Matching Model in Encode Layer revised question generator model shown in FIG. 5, the matching by entering the attention matrix G calculated by the above equation (11) to the RNN matrix M ⁇ R 2d XT is obtained.
  • step S102 as the matching information, a hidden state vector h d0 to the initial state of the Decoder, is generated and match matrix M is a matching model used in the machine reading task.
  • any format such as a vector, a matrix, or a tensor may be used as an expression format of the matching information.
  • a bag-of-words vector in which the word element matched between the input question Q and the related document X is 1, and the other word elements are 0.
  • Information that takes into account the appearance position of the word in the related document X may be used.
  • the matching information is expressed only by a scalar value such as a similarity, information on which part the input question Q and the related document X match is lost. Is preferably not a scalar value.
  • Step S103 The question restoration unit 220 of the revised question generation unit 200 uses the matching information (the hidden state vector hd0 and the matching matrix M) generated by the matching unit 210, the input question Q, and the related document X to be described below. Steps S103-1 to S103-7 generate a natural sentence that becomes the revised question RQ.
  • the word y 0 is assumed to be a token that indicates the beginning of a sentence ⁇ BOS>.
  • steps S103-1 ⁇ step S103-7 will be described for generating a word y s at a certain s.
  • the hidden state vector hd0 calculated in (1) is used.
  • Step S103-2 Next, the question restoration unit 220 uses the attention mechanism (attention) as the processing of the decode layer of the revised question generation model shown in FIG. 5 according to the following equations (12) to (15). Calculate the input z ⁇ s R 3d to the LSTM which is the Decoder.
  • attention mechanism attention
  • W d ⁇ R 2d ⁇ 3d and b d ⁇ R 2d are parameters, and f is an activation function.
  • M t ⁇ R 2d is a column vector composed of elements of the t-th column of the matching matrix M.
  • Step S103-3 Next, the question restoration unit 220 updates the hidden state h ds of the Decoder according to the following equation (16).
  • Step S103-4 Next, the question restoration unit 220 inputs z ⁇ s obtained by the above equation (15) to the LSTM as a Decoder process in the Decode Layer, and calculates a softmax function. As a result, a generation probability distribution P G (y s
  • y ⁇ s , X, Q) is a specific identification that is set in advance as the s-th word y s when the s ⁇ 1th words y s are generated. This is a distribution of conditional probabilities that a word included in the vocabulary set is generated.
  • the specific vocabulary set includes, for example, a set composed of words that appear frequently in a general document.
  • Step S103-5 Next, the question restoration unit 220 uses the weight ⁇ st obtained in the above equation (13) and the softmax function as processing in the Decode Layer, according to the following equation (17): The generation probability P C (y s
  • y ⁇ s , X, Q) is an application of the concept of CopyNet.
  • CopyNet is a neural network model that facilitates generating (copying) an encoded word as it is by giving the word generation probability from outside the LSTM output.
  • a word included in the related document X is generated as the sth word y s by introducing the generation probability P C (y s
  • CopyNet For details of CopyNet, refer to Reference Document 5 and Reference Document 6 below, for example.
  • Step S103-6 Next question restoration unit 220 uses the weighting lambda s, the final generation probability P of a word y s (y s
  • W ⁇ ⁇ R 1 ⁇ 2d and b ⁇ ⁇ R 1 are parameters, and ⁇ is a sigmoid function.
  • y ⁇ s , X, Q) are expressed as P G (y s
  • Q) is a weighted average. For this reason, whether or not the word included in the related document X is copied as y s is determined by the weight ⁇ s .
  • Step S103-7 Next, the question restoration unit 220 generates a word y s based on the final generation probability P (y s
  • the revised question RQ is output by the revised question generation unit 200 to a predetermined output destination.
  • the predetermined output destination include the display device 502, the auxiliary storage device 508, and other programs (for example, a program for executing a question answering task).
  • the revised question RQ is created by adding information in the related information X based on the input question Q.
  • a generation model such as an Encoder-Decoder model using only matching information
  • a revised question RQ that is not very related to the related document X or the input question Q is generated.
  • the input question Q that is regarded as a missing question is used.
  • the revised question RQ related to the related document X can be generated.
  • step S103-7 one word y s is generated for each s.
  • the present invention is not limited to this, and a plurality of words y s are generated for a certain s (or all s). May be.
  • a beam search is a kind of search algorithm such as a breadth-first search of a graph.
  • the question restoration unit 220 When using a beam search, the question restoration unit 220 generates, for example, a word y s for B beam widths for each s.
  • the question restoration unit 220 can generate revised questions RQ of a plurality of variations by outputting the top q items from these candidates in the order of generation score using a beam search.
  • step S103-7 the words y 0 as ⁇ BOS>, a case has been described in which the words in the beginning of a sentence to produce a revised question RQ in the order, not limited to this, for example, a word y
  • the revision question RQ may be generated in order from the word at the end of the sentence, with 0 as ⁇ EOS>.
  • a revised question RQ that compensates for a partial deficiency of the input question Q regarded as a missing question may be generated.
  • a revised question RQ that compensates for all deficiencies in Q may be generated.
  • generating a revised question RQ that compensates for some deficiencies in the input question Q is “partial generation”
  • generating a revised question RQ that compensates for all deficiencies in the input question Q is “general generation”. Represent.
  • a question with clear contents of a question and lack of information necessary for answering (hereinafter, such a question is referred to as “whole question”) is canceled as “plan A is canceled halfway. "What is the charge when doing?" And the input question Q is "What is the charge?”
  • revision question generation process is partial generation or total generation is determined by a learning data set used for the revision question generation model learning process. Whether the revision question generation process is partial generation or total generation is determined according to a question answering task in which the revision question is used.
  • the learning data set is a set of learning data represented by a set of the input question Q used as correct answer data and the related document X.
  • a label that is 1 if the word is a word included in the related document X, and 0 otherwise is given. To do.
  • the input question Q used as correct answer data is represented as “correct answer question Q true ”.
  • FIG. 6 is a flowchart showing an example of the learning process of the revised question generation model in the first embodiment of the present invention.
  • the learning data set is divided into a predetermined number of mini-batches, and the parameters of the revised question generation model are updated for each mini-batch.
  • the following steps S201 to S204 are repeatedly executed using each learning data included in the mini-batch.
  • the following steps S205 to S206 are executed after steps 201 to S204 are executed for all learning data included in the mini-batch.
  • Step S201 The missing question creation unit 300 inputs a correct answer question Q true included in the learning data. Further, the revised question generation model learning unit 400 inputs the correct answer question Q true and the related document X included in the learning data.
  • Step S202 Next, the missing question creation unit 300 creates a question Q (missing question Q) in which a part of the correct question Q true is missing.
  • a question Q missing question Q
  • the missing question creation unit 300 may create all of these missing questions Q, or some (including one) of them.
  • a missing question Q may be created.
  • the missing question creation unit 300 may create the missing question Q for both “tell me the fee” and “tell me”, and either “tell me the fee” or “tell me” A missing question Q may be created.
  • the overall question is "What is the fee for canceling Plan A halfway?”
  • the revised question RQ “What is the fee for canceling midway?” Is generated from the input question Q “What is the fee?”.
  • the revised question RQ “What is the fee for canceling plan A halfway?” Is generated from the input question Q “What is the fee for canceling midway?”.
  • the revised question RQ “ ⁇ BOS>” is generated from the input question Q “What is the fee for canceling plan A halfway?”.
  • Generation of ⁇ BOS> indicates that there is no more clause that can be added (generated). For this reason, it can be known that the second revised question RQ “What is the fee for canceling Plan A halfway?” Is the entire question.
  • any method can be used as a method for creating the missing question Q.
  • a method for creating the missing question Q for example, syntactic analysis such as dependency analysis of the correct answer Q true or phrase structure analysis was performed. Can be created using the results.
  • the granularity of the portion to be deleted from the correct answer question Q true can be set arbitrarily.
  • the missing question creating section 300 is, for example, "the fee at the time of closeout?” That was missing one clause of the beginning of the correct question Q true and, two clauses of the beginning of the correct question Q true Create the missing question Q as “What is the charge?”
  • a method for creating a missing question Q for example, an arbitrary two clauses having a dependency relationship are extracted from a correct question Q true , and a sentence obtained by combining the extracted two clauses according to the dependency relationship is lost. There is a method of using the question Q. At this time, if there is a clause having a dependency relationship with the obtained missing question Q in the correct question Q true , a sentence obtained by combining the missing question Q and the clause may be used as a new missing question Q. .
  • the missing question Q is created by performing phrase structure analysis, dependency tree analysis, etc., and performing loss in sections or words from this analysis result. Just do it.
  • the correct question Q true is English
  • a missing question Q in which a phrase structure below a noun phrase (NP) is missing may be created.
  • the missing question creation unit 300 does not create the missing question Q in which the syntax information of the correct answer question Q true is destroyed. For example, if the correct answer Q true is “Tell me the price for plan A” and use the analysis result of dependency analysis, do not create the missing question Q “Tell me about plan A”, which is not related to dependency. Is preferred.
  • the missing question creation unit 300 may create the missing question Q by pattern matching, for example.
  • the missing position in the correct answer question Q true is determined using a predetermined expression as a marker. Specifically, for example, it is conceivable to use “in the case of” as a marker as a predetermined expression. In this case, if the correct answer Q true is “What is the penalty for contracts of less than 2 years?”, The missing question Q “The penalty for the penalty” Can be created.
  • Step S203 The verification unit 210 of the revised question generation model learning unit 400 generates matching information. Since this step S203 is the same as step S102 by replacing the input question Q in step S102 of FIG. 4 with the missing question Q, description thereof is omitted.
  • Step S204 The question restoration unit 220 of the revised question generation model learning unit 400 generates a revised question RQ. Since this step S204 is the same as step S103 by replacing the input question Q in step S103 of FIG. 4 with the missing question Q, description thereof is omitted.
  • Step S205 The parameter update unit 410 of the revised question generation model learning unit 400 calculates an error between the revised question RQ generated using each learning data included in the mini-batch and the correct question Q true included in the learning data. calculate. For example, cross-entropy may be used as an error function used for error calculation. The error function is appropriately determined according to the revised question generation model.
  • Step S206 The parameter update unit 410 of the revised question generation model learning unit 400 updates the parameters of the revised question generation model using the error calculated in step S205. That is, for example, the parameter update unit 410 calculates the partial differential value of the error function by the error back-propagation method (back propagation) using the error calculated in step S205 described above. Update. Thereby, the revised question generation model is learned.
  • the learning target parameter is expressed as “ ⁇ ” so that each word y s generated with the generation probability P matches the correct question Q true .
  • the generation probability P of the word y s needs to be set to an appropriate ⁇ s as shown in the above equation (18). Therefore, in the first embodiment of the present invention, shall learn the revision question generator model multitask learning for learning the generation probability P and lambda s of words y s simultaneously, the error function is the generation of a word y s
  • the sum L ( ⁇ ) L g + L ⁇ of the error L g related to the probability P and the error L ⁇ related to ⁇ s is assumed.
  • the parameter ⁇ is updated so that the error function L is minimized.
  • lambda s is enough to take a value close to 1 indicates that the probability of the words contained in related document X is copied as y s is increased.
  • a label that is 1 if the word is a word included in the related document X and a label that is 0 otherwise.
  • Shall be granted.
  • ⁇ s is a probability of predicting whether the word y s generated by c ⁇ s is a word included in the related document X, and Become.
  • the revised question RQ is generated, it is determined that the closer ⁇ s is to 1, the higher the probability that the word desired to be generated is in the related document X, and the generation probability P c is strongly considered. It becomes like this.
  • the error L ⁇ can be calculated using binary cross-entropy
  • the error L g can be calculated using negative log likelihood.
  • the revised question generation model shown in FIG. 7 is a model that does not have a mechanism for calculating the generation probability P C (y s
  • y ⁇ s , X, Q) P G (y s
  • y ⁇ s, X, Q) become.
  • the revised question generation model shown in FIG. 8 is a model that does not have a matching layer in addition to the revised question generation model shown in FIG.
  • the attention mechanism calculates the input z ⁇ s to the Decoder using the context matrix H instead of the matching matrix M.
  • FIG. 9 shows a functional configuration of the question generation device 100 that performs the above preprocessing.
  • FIG. 9 is a diagram illustrating a modification of the functional configuration of the question generation device 100 when generating a revised question in the first embodiment of the present invention.
  • the question generation device 100 when generating a revised question may further include a related document search unit 600.
  • the related document search unit 600 inputs the input question Q and the document set Y, and searches the document set Y for a document (related document) X related to the input question Q. Then, the related document search unit 600 outputs the searched related document X to the revised question generation unit 200. Thereby, even when only a document set that is assumed to include the related document X can be obtained, the revised question RQ can be easily obtained.
  • any search method can be used as the search method by the related document search unit 600.
  • N ′ cases with the highest scores are used as the related documents X.
  • the value of N ′ is arbitrarily set, for example, about 1 to 10 can be considered.
  • the question generation device 100 when generating a revised question may further include a display control unit 700.
  • the display control unit 700 displays the related document X searched by the related document search unit 600 and the revised question RQ generated by the revised question generating unit 200 from the related document X and the input question Q.
  • the revised question generator 200 For example, if two related documents X 1 and related documents X 2 are obtained from the document set Y, the revised question generator 200, the revised question RQ 1 using the input question Q and related documents X 1, input question Q And a revised question RQ 2 using the related document X 2 is obtained.
  • the related document search unit 600 of the question generation device 100 reads a plurality of related documents X from the document set Y. Search for (related documents X 1 and related documents X 2). Then, the display control unit 700 of the question generator 100, and related documents X 1 and input question generated by the revised question generator 200 from Q revised question RQ 1 "want to know Price Plan A", related document X 1 and links to, and related documents X 2 and input question Q generated by the revised question generation unit 200 from the revised question RQ 2 "I want to know the price at the time when the special discount is applied", links to related documents X 2 Are displayed to the user (S12). As a result, even when the user makes an ambiguous question (input question Q), the question generating device 100 moves to a plurality of revised questions RQ and related documents X respectively related to the plurality of revised questions RQ. Can be presented to the user.
  • a plurality of revised questions RQ and related documents X may be presented in order.
  • the related document search unit 600 of the question generation device 100 reads a plurality of related documents X from the document set Y. Search for (related documents X 1 and related documents X 2). Then, the display control unit 700 of the question generation device 100 displays, for example, a sentence for confirming to the user whether or not the revised question RQ 1 “I want to know the charges for Plan A” is intended (S22). ).
  • the display control unit 700 of the question generation device 100 may display, for example, the revised question RQ 2 “Special discount applied” A sentence for confirming to the user whether or not the user wants to know the charge when the service is made is displayed (S24).
  • the question generation device 100 interactively links the revised question RQ and the related document X related to the revised question RQ. Can be presented to the user.
  • the question generation device 100 uses, for example, a revised question generation model realized by a neural network, and an input question that may contain a potential defect. From Q, a revised question RQ that does not include a deficit can be generated. Thereby, for example, when a question answering task using the revised question RQ is performed, the answer accuracy of the question answering task can be improved.
  • the revision in which a word included in the related document X related to the input question Q is copied.
  • a question RQ is generated.
  • the question generation device 100 can generate a plurality of variations of revised questions RQ for one input question Q.
  • the question generation device 100 for one input question Q “I want to know the fee”, as the revised question Q, “I want to know the fee for Plan A”, “Special discount” Variations such as “I want to know the fee when is applied” can be generated.
  • the question generation device 100 can, for example, generate a “common question collection (FAQ)”. It can also be applied to automatic creation and expansion.
  • FAQ common question collection
  • N is input to the input question.
  • Number of answers (N is an integer of 1 or more) is generated.
  • the question generation device 100 generates a revised question for each of these N answers.
  • N answers generated by the question answer are candidates for the final answer to the input question (that is, the answer that the questioner really needs), and are also referred to as “answer candidates”.
  • the input question is refined and specified so that the answer can be uniquely determined, and a revised question is generated for each answer.
  • “To dollar” and “To euro” are assigned to the input question, respectively, and revised question 1 “What happens to the yen at 5 pm against the dollar?
  • revised question 2 “What happened to the euro at 5pm against the euro?”.
  • the revised question is generated by the following (1) and (2).
  • a question response is made to the input question, and N answers (answer candidates) for the input question are generated.
  • a revised question for generating the answer is generated (that is, N revised questions corresponding to each of the N answers are generated).
  • the above (1) and (2) can be executed simultaneously end-to-end by the revised question generation model realized by the neural network.
  • the revised question generation model is not necessarily realized by a neural network, and all or a part of the revised question generation model may be realized by a machine learning model other than the neural network.
  • the model that performs the question response of (1) above and the model that generates the revised question of (2) above may be prepared separately and used individually or in combination.
  • an input question used as correct answer data a question in which a part of the input question is missing (that is, a missing question), a related document, And the parameters of the revised question generation model are updated so that the natural sentence obtained using the missing question and the related document approaches the input question which is correct answer data.
  • the revised question generation model similar to the first embodiment, matching between the missing question and the related document is performed, and the missing portion is found from the related document and compensated.
  • the correct answer to the input question is used as the correct answer data, and the parameters of the revised question generation model are updated so that the answer to the input question approaches the correct answer data.
  • FIG. 13 is a diagram illustrating an example of a functional configuration of the question generation device 100 when generating a revised question in the second embodiment of the present invention.
  • the question generation device 100 includes a text processing unit 800, a revised question generation unit 900, and an output unit 1000.
  • the text processing unit 800 inputs an input question and a related document described in a natural sentence, and performs preprocessing for inputting the input question and the related document to the revised question generating unit 900. Specifically, the text processing unit 800 converts an input question and a related document described in a natural sentence into a set of word tokens (word series) by performing, for example, morphological analysis. Note that at least one of the input question and the related document may be a sentence obtained as a speech recognition result.
  • the related document input to the text processing unit 800 may be one or more documents (that is, a set of related documents). In the second embodiment of the present invention, when “related documents” is represented, a set of related documents is also included.
  • This word sequence Q is also expressed as an input question Q.
  • the question generation device 100 may not include the text processing unit 800.
  • the revised question generation unit 900 generates a question response to the input question and a revised question corresponding to the answer (answer candidate) obtained by the question response.
  • the revised question generation unit 900 is realized by a learned revised question generation model (that is, a revised question generation model using parameters updated by a revised question generation model learning unit 1100 described later).
  • the revised question generation unit 900 includes a question response execution unit 910 and a question generation unit 920.
  • the question response execution unit 910 inputs the input question Q and the related document X, performs a question response, and generates answer candidates for the input question Q from the related document X.
  • the number of answer candidates generated here is not necessarily one, and N answer candidates are generated with N being an integer of 1 or more.
  • a method in which the description in the related document is extracted as it is is used as the answer candidate.
  • the present invention is not limited thereto, and a natural sentence question and an arbitrary document (related document) are input Any method can be used as long as a natural sentence answer can be obtained.
  • the question generation unit 920 inputs the input question Q, the related document X, and N answer candidates, and generates a revised question RQ that is more detailed and specific than the input question Q. At this time, the question generation unit 920 generates a revised question RQ for each of the N answer candidates (that is, generates N revised questions RQ corresponding to each of the N answer candidates).
  • the question generation unit 920 generates the revised question RQ by adding information that can uniquely identify each answer candidate to the input question Q.
  • information related to conditions such as “in the case of” and “in the case of” may be described in the vicinity of the information that is the answer candidate in the related document X. Therefore, by adding information regarding such conditions to the input question Q, it is possible to generate a revised question RQ that can uniquely determine an answer (answer candidate) when this condition is met.
  • a proper expression such as a person name or a place name can be useful information for narrowing down answer candidates, and a revised question RQ in which these are added to the input question Q may be generated.
  • the generation method of the revised question RQ the discovery method of information to be added to the input question Q, the method of adding information to the input question Q, and the like are described above. Any method can be employed as long as it can generate a revised question RQ by adding to the input question Q. For example, after finding and extracting the information “in the case of” described above by pattern matching, the information closest to the answer (answer candidate) is added to the head of the input question Q from the extracted information.
  • a method of generating a revised question RQ may be used.
  • the revised question RQ may be generated using a sentence generation method using a neural network.
  • the output unit 1000 outputs N answers (answer candidates) and N revised questions RQ corresponding to each of these N answers. At this time, for example, the output unit 1000 outputs one or more pairs of a certain answer candidate and a revised question RQ corresponding to this answer candidate.
  • a method for outputting a pair of the answer candidate and the revised question RQ an arbitrary method can be adopted according to the user interface of the question generation device 100.
  • the question generation device 100 includes a user interface that outputs an answer to the screen as in a search system or the like, a search result suggestion function for an input question Q input from a user (questioner)
  • the candidate of the revised question RQ is displayed as "Maybe "
  • an answer (answer candidate) corresponding to the revised question RQ is displayed. Also good.
  • the question generating device 100 when the input question Q is input from the user, the revised question RQ corresponding to the most likely answer (answer candidate) is “probably. Say “Yes, is it?” (XX is the question content of the revised question RQ), and utters the answer (answer candidate) corresponding to the revised question RQ when the user agrees You may adopt the method of doing. At this time, for example, when the user disagrees with the confirmation utterance, the user confirms the confirmation question about the revised question RQ corresponding to the next most likely answer (answer candidate). You may adopt the method of repeating this until it agrees.
  • the question generation device 100 may have a function of calculating the likelihood, or the answer candidate is generated together with the generation of the answer candidate by the question answer execution unit 910. Likelihood may be calculated.
  • the output destination of the output unit 1000 is not limited to that described above, and may be, for example, the auxiliary storage device 508, the recording medium 503a, or other devices connected via a network.
  • FIG. 14 is a diagram illustrating an example of a functional configuration of the question generation device 100 during learning according to the second embodiment of the present invention.
  • the question generation device 100 at the time of learning in the second embodiment of the present invention includes a missing question creation unit 300 and a revised question generation model learning unit 1100.
  • the missing question creating unit 300 creates the missing question by inputting the input question Q and missing a part of the input question Q, as in the first embodiment.
  • the revised question generation model learning unit 1100 generates a revised question generation model using the missing question created by the missing question creation unit 300, the input question Q, the correct answer A true for the input question Q, and the related document X. learn. Then, the revised question generation model learning unit 1100 outputs the learned revised question generation model parameters.
  • the revised question generation model learning unit 1100 includes a question response execution unit 910, a question generation unit 920, and a parameter update unit 1110.
  • the question response execution unit 910 and the question generation unit 920 are as described above.
  • the parameter update unit 1110 calculates an error between the natural sentence (revised question RQ) generated by the question generation unit 920 and the input question Q, and answers to the input question Q by the question response execution unit 910 and the input question Q Calculate the error from the correct answer to. Then, using these errors, the parameters of the revised question generation model (revised question generation model parameters that have not been learned) are updated by an arbitrary optimization method.
  • the revised parameter generation model is learned by updating the parameters by the parameter updating unit 1110.
  • the hardware configuration of the question generation device 100 according to the second embodiment of the present invention may be the same as that of the first embodiment, and a description thereof will be omitted.
  • FIG. 15 is a flowchart illustrating an example of a revision question generation process according to the second embodiment of the present invention.
  • the revised question generation process it is assumed that the revised question generation model for realizing the revised question generation unit 900 is realized by a neural network and has been learned.
  • the revised question generation model includes a document encoding layer, a question encoding layer, a document / question matching layer, a machine reading modeling layer, a machine reading output layer, and an answer vector generation. It is a neural network composed of a layer, a decode layer, and a revised question word generation layer.
  • the question response execution unit 910 is realized by the document encoding layer, the question encoding layer, the document / question matching layer, the machine reading modeling layer, and the machine reading output layer.
  • the question generation unit 920 is realized by the answer vector generation layer, the decode layer, and the revised question word generation layer.
  • the document encoding layer, the question encoding layer, the document / question matching layer, and the machine reading modeling layer correspond to the matching unit 210 in the first embodiment.
  • the decode layer and the revised question word generation layer correspond to the question restoration unit 220 in the first embodiment.
  • the neural network that realizes the corrected question generation model in the second embodiment of the present invention includes an encoder-decoder model that is a method for generating a natural sentence in the neural network, and a machine that generates an answer to the question response in the neural network. It is based on a reading model.
  • the description of the answer candidate is directly extracted from the related document X (that is, the position of the start point and the end point when the description is extracted is estimated), thereby generating the answer candidate.
  • This machine reading model is composed of a document / question matching layer, a machine reading modeling layer, and a machine reading output layer.
  • Step S301 The text processing unit 800 inputs an input question described in a natural sentence and a related document.
  • Step S302 The text processing unit 800 converts each input question and related document into a word series. As described above, hereinafter, it is assumed that the input question is converted to the word sequence Q of J word tokens and the related document is converted to the word sequence X of T word tokens. X ".
  • step S302 may not be performed.
  • Step S303 revised question generator 900, the following steps S303-1 ⁇ step S303-3, as the matching information to generate a state vector h q0 and h M0 to the initial state of the decoding layer.
  • Step S303-1 First, the question response execution unit 910 of the revised question generation unit 900 inputs the related document X and the input question Q, and performs processing of the document encoding layer and the question encoding layer of the revised question generation model shown in FIG.
  • the related document X and the input question Q are each converted (encoded) into a d-dimensional word vector sequence. That is, the question response execution unit 910 creates a word vector sequence by converting each word token constituting the related document X and the input question Q into a d-dimensional real vector.
  • the question response execution unit 910 outputs a state vector h q0 when the input question Q is encoded into a d-dimensional word vector sequence.
  • the word vector series of the related document X is expressed as “document vector series H” as H.
  • the word vector series of the input question Q is expressed as “question vector series U” as represented by U.
  • the document vector sequence is H ⁇ R d ⁇ T
  • the query vector sequence is U ⁇ R d ⁇ J.
  • any method can be adopted as long as the document vector sequence and the question vector sequence can be generated.
  • a method of inputting a related document X and an input question Q to a word embedding layer (Word Embedding Layer) and converting each word token into a d-dimensional real vector, and then converting the word token into a word vector sequence by RNN is used.
  • Word Embedding Layer word embedding layer
  • encoding using an attention mechanism (attention) may be performed.
  • the decoding layer uses the state vector h q0 output from the question encoding layer as an initial state, it is necessary to generate the state vector h q0 by an arbitrary method.
  • the state vector h q0 is generated only in the question encoding layer.
  • the state vector h x0 may be generated only in the document encoding layer or in the document encoding layer.
  • the decoding layer may use the state vector h x0 as an initial state.
  • the decoding layer uses one or both of these state vectors as the initial state. Can do.
  • Step S303-2 Next, the question response execution unit 910 of the revised question generation unit 900 uses the document vector series H and the question vector series U as processing of the document / question matching layer of the revised question generation model shown in FIG. Thus, information related to the input question Q is found and extracted in the related document X for machine reading. This discovery and extraction is performed by collating the related document X with the input question Q.
  • any method can be adopted.
  • BiDAF using an attention mechanism can be employed.
  • QANet using CNN Convolutional Neural Network
  • QANet using CNN Convolutional Neural Network
  • Step S303-3 The question answer execution unit 910 of the revised question generation unit 900 uses the matching vector sequence G as a machine reading modeling layer process of the revised question generation model shown in FIG. R d ⁇ T is created.
  • the machine-reading modeling vector sequence M is created by performing a technique using RNN on the collation vector sequence G, for example, as in the document encoding layer and the question encoding layer.
  • the question response execution unit 910 generates a hidden state vector h M0 in the same manner as the question encoding layer. This hidden state vector h M0 is used as the initial state of the decode layer.
  • the machine reading modeling vector series M corresponds to the matching matrix M in the first embodiment.
  • Step S304 Next, the question answer execution unit 910 of the revised question generation unit 900 generates answer candidates by using the machine reading modeling vector sequence M as the process of the machine reading output layer of the revised question generation model shown in FIG. To do.
  • the generation of the answer candidates is performed by extracting the start point and the end point of the description as the answer candidate from the related document X.
  • the machine reading modeling vector sequence M is linearly converted with weights W 0 ⁇ R 1 ⁇ d. after having created the starting point vector O start ⁇ R T by, it converted into a probability distribution P start by applying the softmax function sequence length T with respect to the starting point vector O start. Then, using this probability distribution P start , the t start (0 ⁇ t start ⁇ T) -th element with the highest probability is extracted from the related document X and used as the starting word.
  • the start point vector O start and the machine reading modeling vector sequence M are input to the RNN and new.
  • a machine reading modeling vector series M ′ is created.
  • a probability distribution P end is obtained from the new machine-reading modeling vector sequence M ′ by the same method as the starting point, and t end (t start ⁇ t end ⁇ T) th with the highest probability is obtained using this probability distribution P end.
  • P (i, k) P start (i) ⁇ P end (k) is first calculated using P start and P end . However, 0 ⁇ i ⁇ T and i ⁇ k ⁇ T. Then, a combination of i, k with the top N P (i, k) may be used as the start point and the end point. As a result, the sections corresponding to the top N i, k combinations are respectively extracted as N answers (answer candidates).
  • the question answer execution unit 910 may output the start point and the end point of each of N answers (answer candidates), may output the N answers (answer candidates) themselves, or N The start point word and end point word of each answer (answer candidate) may be output.
  • the subsequent step S305 is executed for each of the N start point and end point sets.
  • a certain set of start point t start and end point t end is set as “answer candidate A”. Step S305 will be described for the answer candidate A.
  • Step S305 The revised question generation unit 900 generates a revised question corresponding to the answer candidate A through the following steps S305-1 to S305-3.
  • Step S305-1 The question generation unit 920 of the revised question generation unit 900 inputs the answer candidate A (that is, the start point t start and the end point t end ), and processes the response vector generation layer of the revised question generation model shown in FIG. As an answer vector corresponding to answer candidate A
  • d a represents the number of dimensions of the answer vector.
  • any method can be adopted as long as the answer vector a can be created using the answer candidate A (that is, the start point t start and the end point t end ) as an input.
  • the description of the section from the start point t start to the end point t end is once converted into a word sequence, and this word sequence is converted into a vector by the document encoding layer, and the answer vector a may be used.
  • the start point t start and the end point section H (t start, t end) which is determined by t end the RNN respect ⁇ R d ⁇ l (l is the sequence length of the answer candidate a) vector sequences was extracted from the document vector sequence, corresponding to the extracted sections
  • the answer vector a may be created by applying or calculating the center of gravity vector.
  • a method for generating a sentence to be an answer (answer candidate A) with reference to the description in the related document X instead of using the answer (answer candidate A) extracted as it is in the related document X Is used, the generated sentence (the sentence that becomes the answer) is used as an input, and the answer vector a may be created as the process of the answer vector generation layer.
  • Step S305-2 The question generation unit 920 of the revised question generation unit 900 outputs the words constituting the revised question using the answer vector a by the RNN as processing of the decoding layer of the revised question generation model shown in FIG. Create a vector to do.
  • the state vectors h q0 and h M0 output from the question response execution unit 910 are used as initial values (initial states) of the state vectors.
  • the RNN may be two layers, and the initial state of the first layer RNN may be h q0 , and the initial state of the second layer RNN may be h M0 .
  • an average vector of two state vectors h q0 and h M0 may be set as an initial state, Only one of the two state vectors h q0 and h M0 may be set as the initial state.
  • the state vector h x0 of the document encoding layer may be used to determine the initial state of the decoding layer for the state vectors h q0 and h x0 .
  • the state vector h x0 of the document encoding layer may be used to determine the initial state of the decoding layer for the state vectors h q0 and h x0 .
  • the decoding layer contains the embedded vector of the previously generated word.
  • d e represents the number of dimensions of the word embedding vectors.
  • a vector in which a response vector is combined with a word embedding vector is combined with a word embedding vector.
  • any technique used in the decoding layer of the Encoder-Decoder model such as attention mechanism or copying, may be applied to the decoding layer of the revised question generation model shown in FIG.
  • Step S305-3 The question generation unit 920 of the revision question generation unit 900 generates the s-th word y s constituting the revision question from the output of the decode layer, similarly to the Encoder-Decoder model. That is, for example, after linearly converting the output result of the decoding layer, the word generation probability in the related document X is generated by the softmax function. Then, for example, words, word generation probability becomes the maximum, is generated as s-th word y s. By repeating this until the word y s is ⁇ EOS> is generated, the words constituting the revised candidate corresponding to the answer candidate A is generated. It should be noted, y 0 is assumed to be ⁇ BOS>.
  • Step S306 Finally, the output unit 1000 outputs N answers (answer candidates) and N revised questions RQ corresponding to each of these N answers.
  • FIG. 17 is a flowchart showing an example of the learning process of the revised question generation model in the second embodiment of the present invention.
  • a machine-reading corpus is used to learn the revised question generation model.
  • the machine-reading corpus includes a plurality of sets of “question”, “document to be questioned”, and “answer range (or character string of the answer range) in the question target document”.
  • the “document to be questioned” included in the corpus is the related document X
  • the “question” included in the corpus is the input question Q
  • the correct answer A true for the input question Q is the “question” in the corpus.
  • the response range (or character string of the response range) in the target document is used as it is.
  • the input question Q and the correct answer A true of the answer to the input question Q are used as learning data for machine reading processing in the question response execution unit 910.
  • the correct answer A true of the answer is represented by a set of a start point and an end point.
  • Step S401 The text processing unit 800 inputs a plurality of learning data (that is, a learning data set) and related documents.
  • Step S402 The text processing unit 800 converts a plurality of input questions and related documents respectively included in the plurality of input learning data into a plurality of input questions Q and a related document X that are word sequences.
  • a plurality of input questions and related documents that have been input are often already expressed in a word sequence, and therefore this step S402 need not be performed.
  • the learning data set is divided into a predetermined number of mini-batches, and the parameters of the revised question generation model are updated for each mini-batch.
  • steps S403 to S406 are repeatedly executed using each learning data included in the mini-batch.
  • steps S407 to S409 are executed after steps 401 to S206 are executed for all the learning data included in the mini-batch.
  • Step S403 The missing question creation unit 300 creates a question Q (missing question Q) in which a part of the input question Q that is learning data is missing. Since the input question Q is correct answer data for the missing question Q, the input question Q is hereinafter referred to as a correct question Q true .
  • the missing question Q may be statistically created using the trained Encoder-Decoder model, or the missing question Q is created by cutting off clauses and phrases using syntax information such as sentence dependency. May be.
  • the missing question Q may be created using a sentence compression technique that is one of the tasks of natural language processing.
  • Step S404 The question response execution unit 910 of the revised question generation model learning unit 1100 generates matching information. Since this step S404 is the same as step S303 by replacing the input question Q in step S303 of FIG. 15 with the missing question Q, the description thereof is omitted.
  • Step S405 The question response execution unit 910 of the revised question generation model learning unit 1100 generates answer candidates for the missing question Q.
  • This step S405 is the same as step S304 by replacing the input question Q in step S304 of FIG.
  • Step S406 The question generation unit 920 of the revised question generation model learning unit 1100 generates a revised question RQ corresponding to each answer candidate of the missing question Q. This step S406 is the same as step S305 by replacing the input question Q in step S305 of FIG.
  • Step S407 The parameter update unit 1110 of the revised question generation model learning unit 1100 generates the revised question RQ generated using each learning data included in the mini-batch and the input question Q (that is, correct answer question) included in the learning data. Calculate the first error of Q true ).
  • the parameter updating unit 1110 calculates a second error between the answer A to the input question Q included in each learning data included in the mini-batch and the correct answer A true included in the learning data.
  • the answer A is obtained as an answer in the question answer by inputting the input question Q (and the related document X) to the question answer execution unit 910.
  • cross-entropy may be used as the error function used for calculating the first error and the second error.
  • the error function is appropriately determined according to the revised question generation model.
  • Step S408 The parameter update unit 1110 of the revised question generation model learning unit 1100 updates the parameters of the revised question generation model using the first error and the second error calculated in Step S407. That is, for example, the parameter update unit 410 calculates the partial differential value of the error function by the error back propagation method (back propagation) using the first error and the second error calculated in step S407 above. The parameter of the revised question generation model is updated. Thereby, the revised question generation model is learned.
  • correct answer data is generated by each of the machine reading (that is, the question answer execution unit 910) and the revised question generation (that is, the question generation unit 920). That is, an error function is defined for the correct answer question Q true for the revised question RQ and the correct answer A true for the correct question Q true, and the sum of these error function values (ie, the first error and the second error). ) Is treated as an error of the entire neural network, and the parameter is updated so that this error is reduced (that is, the parameter is updated by multitask learning).
  • the question generation device 100 uses the revised question generation model realized by, for example, a neural network, and generates the revised question RQ with respect to the input question Q.
  • a question answer is performed, and a revised question RQ corresponding to the answer candidate obtained by this question answer is generated.
  • a revised question RQ is generated for each answer candidate, so by using these revised questions RQ in the question answering task, , It will be possible to achieve high response accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

This invention is characterized by comprising a generation means for receiving input of question text and a related document including an answer to the question text and using a pre-learned machine learning model to generate revised question text wherein a potentially missing portion of the question text is supplemented with words which are included in a prescribed vocabulary set.

Description

質問生成装置、質問生成方法及びプログラムQuestion generating apparatus, question generating method and program
　本発明は、質問生成装置、質問生成方法及びプログラムに関する。 The present invention relates to a question generation device, a question generation method, and a program.
　近年、スマートフォンやスマートスピーカ等のデバイス上で、ユーザが自然言語で入力した質問に対する回答をコンピュータが自動で行う質問応答技術が注目を集めている。このような質問応答技術として、自然言語で入力された質問に対して、同じく自然言語で記述された文書内から回答となる部分を抽出する機械読解型の質問応答技術が知られている（例えば、非特許文献１参照）。 In recent years, a question answering technique in which a computer automatically answers a question input by a user in a natural language on a device such as a smartphone or a smart speaker has attracted attention. As such a question answering technique, a machine reading type question answering technique for extracting a part which becomes an answer from a document written in a natural language for a question inputted in a natural language is known (for example, Non-Patent Document 1).
　機械読解型の質問応答技術では、ニューラルネットワークを用いて、質問と、マニュアル等の文書に記述されている回答部分との照合を行っており、人と同等以上の回答精度を達成できることが知られている。 Machine reading comprehension type question answering technology uses a neural network to collate the question with the answer part described in a manual or other document, and is known to be able to achieve answer accuracy equivalent to or better than that of humans. ing.
　ここで、機械読解型の質問応答技術において高い回答精度を達成するためには、質問内容が明確であり、かつ、回答に必要な情報が不足なく質問に含まれている必要がある。しかしながら、機械読解型の質問応答技術を用いた実際のサービスでは、質問内容が曖昧であったり、質問文が短すぎたりする場合がある。このような場合、質問に対する回答が一意に決定できなかったり、回答内容を間違えたりする可能性があり、高い回答精度が達成できないことがある。 Here, in order to achieve high answer accuracy in the machine-reading-type question answering technology, the question content must be clear and the information necessary for answering must be included in the question without any shortage. However, in an actual service using a machine-reading-type question answering technique, the question content may be ambiguous or the question sentence may be too short. In such a case, there is a possibility that the answer to the question cannot be uniquely determined or the answer content may be wrong, and high answer accuracy may not be achieved.
　本発明の一実施形態は、上記の点に鑑みてなされたもので、質問に対する高い回答精度を実現することを目的とする。 An embodiment of the present invention has been made in view of the above points, and an object thereof is to achieve high accuracy in answering a question.
　上記目的を達成するため、本発明の一実施形態における質問生成装置は、質問文と、該質問文に対する回答が含まれる関連文書とを入力とし、予め学習済みの機械学習モデルを用いて、前記質問文の潜在的に欠損している部分を、所定の語彙集合に含まれる単語で補った改訂質問文を生成する生成手段、を有することを特徴とする。 In order to achieve the above object, a question generation device according to an embodiment of the present invention receives a question sentence and a related document including an answer to the question sentence as input, and uses a machine learning model that has been learned in advance, And generating means for generating a revised question sentence in which a potentially missing part of the question sentence is supplemented with a word included in a predetermined vocabulary set.
　質問に対する高い回答精度を実現することができる。 高 い High accuracy of answers to questions can be realized.
本発明の第一の実施形態における改訂質問生成時の質問生成装置の機能構成の一例を示す図である。It is a figure which shows an example of a function structure of the question production | generation apparatus at the time of the revision question production | generation in 1st embodiment of this invention. 本発明の第一の実施形態における学習時の質問生成装置の機能構成の一例を示す図である。It is a figure which shows an example of a function structure of the question generation apparatus at the time of learning in 1st embodiment of this invention. 本発明の第一の実施形態における質問生成装置のハードウェア構成の一例を示す図である。It is a figure which shows an example of the hardware constitutions of the question generation apparatus in 1st embodiment of this invention. 本発明の第一の実施形態における改訂質問の生成処理の一例を示すフローチャートである。It is a flowchart which shows an example of the production | generation process of the revision question in 1st embodiment of this invention. 本発明の第一の実施形態における改訂質問生成モデルをニューラルネットワークで実現した場合の一例を示す図である。It is a figure which shows an example at the time of implement | achieving the revision question generation model in 1st embodiment of this invention with the neural network. 本発明の第一の実施形態における改訂質問生成モデルの学習処理の一例を示すフローチャートである。It is a flowchart which shows an example of the learning process of the revision question production | generation model in 1st embodiment of this invention. 本発明の第一の実施形態における改訂質問生成モデルをニューラルネットワークで実現した場合の変形例（その１）を示す図である。It is a figure which shows the modification (the 1) at the time of implement | achieving the revision question production | generation model in 1st embodiment of this invention with a neural network. 本発明の第一の実施形態における改訂質問生成モデルをニューラルネットワークで実現した場合の変形例（その２）を示す図である。It is a figure which shows the modification (the 2) at the time of implement | achieving the revision question production | generation model in 1st embodiment of this invention with a neural network. 本発明の第一の実施形態における改訂質問生成時の質問生成装置の機能構成の変形例を示す図である。It is a figure which shows the modification of the function structure of the question generation apparatus at the time of the revision question generation in 1st embodiment of this invention. チャットボットへの応用例（その１）を示す図である。It is a figure which shows the application example (the 1) to a chat bot. チャットボットへの応用例（その２）を示す図である。It is a figure which shows the application example (the 2) to a chat bot. 本発明の第二の実施形態における改訂質問の一例を説明するための図である。It is a figure for demonstrating an example of the revision question in 2nd embodiment of this invention. 本発明の第二の実施形態における改訂質問生成時の質問生成装置の機能構成の一例を示す図である。It is a figure which shows an example of a function structure of the question production | generation apparatus at the time of the revision question production | generation in 2nd embodiment of this invention. 本発明の第二の実施形態における学習時の質問生成装置の機能構成の一例を示す図である。It is a figure which shows an example of a function structure of the question generation apparatus at the time of learning in 2nd embodiment of this invention. 本発明の第二の実施形態における改訂質問の生成処理の一例を示すフローチャートである。It is a flowchart which shows an example of the production | generation process of the revision question in 2nd embodiment of this invention. 本発明の第二の実施形態における改訂質問生成モデルをニューラルネットワークで実現した場合の一例を示す図である。It is a figure which shows an example at the time of implement | achieving the revision question production | generation model in 2nd embodiment of this invention with a neural network. 本発明の第二の実施形態における改訂質問生成モデルの学習処理の一例を示すフローチャートである。It is a flowchart which shows an example of the learning process of the revision question production | generation model in 2nd embodiment of this invention.
　以下、本発明の各実施形態について、図面を参照しながら詳細に説明する。以降では、機械読解型の質問応答技術を用いた質問応答の回答精度を高めることを目的として、入力された質問（以降、単に「入力質問」とも表す。）の改訂質問（ＲＱ：Revised Question）を生成する質問生成装置１００について説明する。改訂質問とは、入力質問の質問内容を補強した、より具体的な内容の質問文のことである。すなわち、改訂質問とは、質問内容が明確であり、かつ、回答に必要な情報が不足なく含まれている質問のことである。 Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following, the revised question (RQ: Revised Question) of the input question (hereinafter, also simply referred to as “input question”) for the purpose of improving the accuracy of answering the question response using the machine-reading-type question answering technology. The question generation device 100 that generates The revised question is a question sentence with more specific contents that reinforces the question contents of the input question. That is, the revised question is a question in which the content of the question is clear and the information necessary for the answer is included without a shortage.
　質問に対する回答を生成及び応答するタスク（質問応答タスク）の前に、当該質問の改訂質問を生成した上で、当該改訂質問を用いた質問応答タスクを行うことで、質問応答の回答精度を高めることができるようになる。 Before the task for generating and responding to an answer to a question (question answering task), the revised question of the question is generated, and then the question answering task using the revised question is performed to improve the answer accuracy of the question answer. Will be able to.
　なお、以下で説明する各実施形態は一例に過ぎず、本発明を適用可能な形態は、以下の各実施形態に限定されない。本発明の各実施形態に係る技術は、例えば、ユーザが自然言語で入力した質問に対する回答を提供するサービス等に利用可能であるが、利用対象はこれに限られず、様々な対象に利用可能である。 In addition, each embodiment described below is only an example, and forms to which the present invention can be applied are not limited to the following embodiments. The technology according to each embodiment of the present invention can be used for, for example, a service that provides an answer to a question input by a user in a natural language, but the usage target is not limited to this and can be used for various targets. is there.
　［第一の実施形態］
　まず、本発明の第一の実施形態について説明する。 [First embodiment]
First, a first embodiment of the present invention will be described.
　（概要）
　本発明の第一の実施形態では、入力質問と、この入力質問に関連する文書（以降、「関連文書」とも表す。）とが与えられた場合に、質問生成装置１００が、改訂質問を生成する機械学習モデル（以降、「改訂質問生成モデル」とも表す。）を用いて、当該入力質問の改訂質問を生成する。 (Overview)
In the first embodiment of the present invention, when an input question and a document related to the input question (hereinafter also referred to as “related document”) are given, the question generation device 100 generates a revised question. The revised question of the input question is generated using a machine learning model (hereinafter also referred to as a “revised question generation model”).
　より具体的には、本発明の第一の実施形態では、改訂質問生成モデルにより、入力質問と関連文書とのマッチングを行って、入力質問の潜在的に欠損した部分（単語や文節等の文字列）を補うことで、改訂質問を生成する。これにより、例えば、質問内容が曖昧な入力質問や質問文が短すぎる入力質問が与えられた場合に、入力質問よりも詳細化・具体化した改訂質問が生成される。また、このとき、関連文書を用いて改訂質問が生成されることにより、例えば、質問応答タスクを行うシステムが回答可能な改訂質問を生成することができる（言い換えれば、質問応答タスクを行うシステムが回答不能な改訂質問は生成されないようにすることができる。）。 More specifically, in the first embodiment of the present invention, a revised question generation model is used to match an input question with a related document, and a potentially missing part of the input question (characters such as words and phrases). A revised question is generated by supplementing the column. Thereby, for example, when an input question with ambiguous question content or an input question with a too short question text is given, a revised question that is more detailed and specific than the input question is generated. At this time, the revised question is generated using the related document, so that, for example, a revised question that can be answered by the system that performs the question answering task can be generated (in other words, the system that performs the question answering task is Unanswered revised questions can be prevented from being generated.)
　また、本発明の第一の実施形態では、正解データとして用いる入力質問と、この入力質問の一部を欠損させた質問（これを「欠損質問」とも表す。）と、関連文書とを用いて、改訂質問生成モデルを学習する。この学習では、欠損質問と関連文書とを用いて得られる自然文が、正解データである入力質問に近付くように改訂質問生成モデルのパラメータを更新する。欠損質問とは、入力された関連文書に関する質問文として、必要な情報（単語や文節等の文字列）が一部欠けている質問文のことである。なお、自然文とは、自然言語で記述された文のことである。 In the first embodiment of the present invention, an input question used as correct answer data, a question in which a part of the input question is missing (also referred to as a “missing question”), and a related document are used. Learn the revised question generation model. In this learning, the parameters of the revised question generation model are updated so that the natural sentence obtained using the missing question and the related document approaches the input question that is correct answer data. A missing question is a question sentence in which necessary information (a character string such as a word or a phrase) is partially missing as a question sentence related to an input related document. A natural sentence is a sentence written in a natural language.
　ここで、本発明の第一の実施形態では、入力質問は、自然言語で記述された文（すなわち、自然文）であり、例えば形態素解析等を行うことによってＪ個の単語トークンの集合Ｑ＝｛ｑ０，ｑ１，・・・，ｑＪ－１｝と表されるものとする。なお、入力質問となる文は自然文以外にも、例えば、単にキーワードを列挙しただけの文でも良い。また、音声認識結果として得られた文等であっても良い。 Here, in the first embodiment of the present invention, the input question is a sentence written in a natural language (that is, a natural sentence). For example, by performing morphological analysis or the like, a set of J word tokens Q = It is assumed that {q 0 , q 1 ,..., Q J−1 }. In addition, the sentence used as an input question may be, for example, a sentence in which keywords are simply listed in addition to a natural sentence. Moreover, the sentence etc. which were obtained as a speech recognition result may be sufficient.
　また、関連文書は、例えば数百語程度の単語から構成された文であり、Ｔ個の単語トークンの集合Ｘ＝｛ｘ０，ｘ１，・・・，ｘＴ－１｝と表されるものとする。ここで、関連文書には、入力質問に対する回答となる情報が含まれるものとする。関連文書としては、例えば、入力質問に対する回答が記述されたマニュアル等が挙げられる。なお、本発明の第一の実施形態では、関連文書をパッセージ（Passage）とも称する。 The related document is a sentence composed of, for example, about several hundred words, and is expressed as a set of T word tokens X = {x 0 , x 1 ,..., X T−1 }. Shall. Here, it is assumed that the related document includes information serving as an answer to the input question. Examples of the related document include a manual in which an answer to an input question is described. In the first embodiment of the present invention, the related document is also referred to as a passage.
　また、改訂質問は、入力質問が詳細化・具体化された文であり、Ｓ個の単語トークンの集合ＲＱ＝｛ｙ０，ｙ１，・・・，ｙＳ－１｝と表されるものとする。 The revised question is a sentence in which the input question is refined and materialized, and is expressed as a set of S word tokens RQ = {y 0 , y 1 ,..., Y S−1 }. And
　（質問生成装置１００の機能構成）
　まず、本発明の第一の実施形態における改訂質問生成時の質問生成装置１００の機能構成について、図１を参照しながら説明する。図１は、本発明の第一の実施形態における改訂質問生成時の質問生成装置１００の機能構成の一例を示す図である。 (Functional configuration of the question generation device 100)
First, the functional configuration of the question generation device 100 when generating a revised question in the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a functional configuration of a question generating device 100 at the time of generating a revised question in the first embodiment of the present invention.
　図１に示すように、本発明の第一の実施形態における改訂質問生成時の質問生成装置１００は、改訂質問生成部２００を有する。改訂質問生成部２００は、学習済みの改訂質問生成モデル（すなわち、後述する改訂質問生成モデル学習部４００によって更新されたパラメータを用いた改訂質問生成モデル）により実現される。 As shown in FIG. 1, the question generation device 100 for generating a revised question in the first embodiment of the present invention includes a revised question generation unit 200. The revised question generation unit 200 is realized by a learned revised question generation model (that is, a revised question generation model using parameters updated by a revised question generation model learning unit 400 described later).
　改訂質問生成部２００は、質問（入力質問）と関連文書とを入力して、改訂質問を生成及び出力する。より具体的には、改訂質問生成部２００は、入力質問を欠損質問と見做した上で、関連文書を用いて、欠損する前の質問文を復元することで、改訂質問を生成する。 The revised question generation unit 200 inputs a question (input question) and a related document, and generates and outputs a revised question. More specifically, the revised question generation unit 200 generates a revised question by regarding the input question as a missing question and restoring the previous question text using the related document.
　ここで、改訂質問生成部２００には、照合部２１０と、質問復元部２２０とが含まれる。照合部２１０は、入力質問と関連文書とのマッチング情報を生成する。マッチング情報とは、入力質問に含まれる各単語と関連文書に含まれる各単語との一致関係を表す情報である。質問復元部２２０は、照合部２１０が生成したマッチング情報と、入力質問と、関連文書とを用いて、入力質問が、欠損する前の質問文となるように自然文を生成（復元）する。質問復元部２２０により生成された自然文が改訂質問となる。 Here, the revised question generation unit 200 includes a collation unit 210 and a question restoration unit 220. The collation unit 210 generates matching information between the input question and the related document. The matching information is information representing a matching relationship between each word included in the input question and each word included in the related document. The question restoration unit 220 uses the matching information generated by the matching unit 210, the input question, and the related document to generate (restore) a natural sentence so that the input question becomes a question sentence before being lost. The natural sentence generated by the question restoration unit 220 becomes a revised question.
　次に、本発明の第一の実施形態における学習時の質問生成装置１００の機能構成について、図２を参照しながら説明する。図２は、本発明の第一の実施形態における学習時の質問生成装置１００の機能構成の一例を示す図である。 Next, the functional configuration of the question generation device 100 during learning in the first embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a functional configuration of the question generation device 100 during learning according to the first embodiment of the present invention.
　図２に示すように、本発明の第一の実施形態における学習時の質問生成装置１００は、欠損質問作成部３００と、改訂質問生成モデル学習部４００とを有する。 As shown in FIG. 2, the question generation device 100 at the time of learning in the first embodiment of the present invention includes a missing question creation unit 300 and a revised question generation model learning unit 400.
　欠損質問作成部３００は、質問（入力質問）を入力して、入力質問の一部を欠損させることで、欠損質問を作成する。 The missing question creation unit 300 creates a missing question by inputting a question (input question) and missing a part of the input question.
　改訂質問生成モデル学習部４００は、欠損質問作成部３００が作成した欠損質問と、入力質問と、関連文書とを用いて、改訂質問生成モデルを学習する。そして、改訂質問生成モデル学習部４００は、学習済みの改訂質問生成モデルのパラメータを出力する。 The revised question generation model learning unit 400 learns a revised question generation model using the missing question created by the missing question creation unit 300, the input question, and the related document. Then, the revised question generation model learning unit 400 outputs the learned parameters of the revised question generation model.
　ここで、改訂質問生成モデル学習部４００には、照合部２１０と、質問復元部２２０と、パラメータ更新部４１０とが含まれる。照合部２１０及び質問復元部２２０は、上述した通りである。パラメータ更新部４１０は、質問復元部２２０が生成した自然文（改訂質問）と、入力質問との誤差を算出した上で、この誤差を用いて、任意の最適化方法により改訂質問生成モデルのパラメータ（学習済みでない改訂質問生成モデルパラメータ）を更新する。パラメータ更新部４１０によりパラメータが更新されることで、改訂質問生成モデルが学習される。 Here, the revised question generation model learning unit 400 includes a verification unit 210, a question restoration unit 220, and a parameter update unit 410. The collation unit 210 and the question restoration unit 220 are as described above. The parameter update unit 410 calculates an error between the natural sentence (revised question) generated by the question restoration unit 220 and the input question, and uses the error to change the parameter of the revised question generation model by an arbitrary optimization method. (Revised question generation model parameter not learned) is updated. The revised parameter generation model is learned by updating the parameters by the parameter updating unit 410.
　本発明の第一の実施形態では、改訂質問生成モデルは、ニューラルネットワークで実現された機械学習モデルであるものとする。ただし、改訂質問生成モデルの全部又は一部が、ニューラルネットワーク以外の機械学習モデルで実現されていても良い。例えば、照合部２１０及び質問復元部２２０のうちの少なくとも一方の機能部が、ニューラルネットワーク以外の機械学習モデルで実現されていても良い。 In the first embodiment of the present invention, the revised question generation model is a machine learning model realized by a neural network. However, all or part of the revised question generation model may be realized by a machine learning model other than the neural network. For example, at least one functional unit of the matching unit 210 and the question restoration unit 220 may be realized by a machine learning model other than the neural network.
　（質問生成装置１００のハードウェア構成）
　次に、本発明の第一の実施形態における質問生成装置１００のハードウェア構成について、図３を参照しながら説明する。図３は、本発明の第一の実施形態における質問生成装置１００のハードウェア構成の一例を示す図である。 (Hardware configuration of question generation device 100)
Next, the hardware configuration of the question generation device 100 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of a hardware configuration of the question generation device 100 according to the first embodiment of the present invention.
　図３に示すように、本発明の第一の実施形態における質問生成装置１００は、入力装置５０１と、表示装置５０２と、外部Ｉ／Ｆ５０３と、ＲＡＭ（Random Access Memory）５０４と、ＲＯＭ（Read Only Memory）５０５と、演算装置５０６と、通信Ｉ／Ｆ５０７と、補助記憶装置５０８とを有する。これら各ハードウェアは、それぞれがバスＢを介して通信可能に接続されている。 As shown in FIG. 3, the question generation device 100 according to the first embodiment of the present invention includes an input device 501, a display device 502, an external I / F 503, a RAM (Random Access Memory) 504, and a ROM (Read Only Memory) 505, an arithmetic device 506, a communication I / F 507, and an auxiliary storage device 508. Each of these hardware is connected via a bus B so as to be able to communicate.
　入力装置５０１は、例えばキーボードやマウス、タッチパネル等であり、ユーザが各種操作を入力するのに用いられる。表示装置５０２は、例えばディスプレイ等であり、質問生成装置１００の処理結果（例えば、改訂質問等）を表示する。なお、質問生成装置１００は、入力装置５０１及び表示装置５０２の少なくとも一方を有していなくても良い。 The input device 501 is, for example, a keyboard, a mouse, a touch panel, or the like, and is used by a user to input various operations. The display device 502 is a display or the like, for example, and displays a processing result (for example, a revised question or the like) of the question generation device 100. The question generation device 100 may not include at least one of the input device 501 and the display device 502.
　外部Ｉ／Ｆ５０３は、外部装置とのインタフェースである。外部装置には、記録媒体５０３ａ等がある。質問生成装置１００は、外部Ｉ／Ｆ５０３を介して、記録媒体５０３ａ等の読み取りや書き込み等を行うことができる。記録媒体５０３ａには、質問生成装置１００が有する各機能部を実現する１以上のプログラム等が記録されていても良い。 External I / F 503 is an interface with an external device. The external device includes a recording medium 503a and the like. The question generation device 100 can read and write the recording medium 503a and the like via the external I / F 503. The recording medium 503a may store one or more programs that realize each functional unit included in the question generation device 100.
　記録媒体５０３ａには、例えば、フレキシブルディスク、ＣＤ（Compact Disc）、ＤＶＤ（Digital Versatile Disk）、ＳＤメモリカード（Secure Digital memory card）、ＵＳＢ（Universal Serial Bus）メモリカード等がある。 Examples of the recording medium 503a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), a USB (Universal Serial Bus) memory card, and the like.
　ＲＡＭ５０４は、プログラムやデータを一時保持する揮発性の半導体メモリである。ＲＯＭ５０５は、電源を切ってもプログラムやデータを保持することができる不揮発性の半導体メモリである。ＲＯＭ５０５には、例えば、ＯＳ（Operating System）に関する設定や通信ネットワークに関する設定等が格納されている。 The RAM 504 is a volatile semiconductor memory that temporarily stores programs and data. The ROM 505 is a nonvolatile semiconductor memory that can retain programs and data even when the power is turned off. The ROM 505 stores, for example, settings related to an OS (Operating System), settings related to a communication network, and the like.
　演算装置５０６は、例えば、ＣＰＵ（Central Processing Unit）やＧＰＵ（Graphics Processing Unit）等であり、ＲＯＭ５０５や補助記憶装置５０８等からプログラムやデータをＲＡＭ５０４上に読み出して処理を実行する。質問生成装置１００が有する各機能部は、例えば、補助記憶装置５０８に格納されている１以上のプログラムが演算装置５０６に実行させる処理により実現される。なお、質問生成装置１００は、演算装置５０６としてＣＰＵとＧＰＵとの両方を有していても良いし、ＣＰＵ又はＧＰＵのいずれか一方のみを有していても良い。 The computing device 506 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and reads a program or data from the ROM 505, the auxiliary storage device 508, or the like onto the RAM 504 and executes processing. Each functional unit included in the question generation device 100 is realized by, for example, processing that the arithmetic device 506 causes one or more programs stored in the auxiliary storage device 508 to execute. The question generation device 100 may include both the CPU and the GPU as the arithmetic device 506, or may include only one of the CPU and the GPU.
　通信Ｉ／Ｆ５０７は、質問生成装置１００を通信ネットワークに接続するためのインタフェースである。質問生成装置１００が有する各機能部を実現する１以上のプログラムは、通信Ｉ／Ｆ５０７を介して、所定のサーバ装置等から取得（ダウンロード）されても良い。 The communication I / F 507 is an interface for connecting the question generating device 100 to a communication network. One or more programs that realize each functional unit included in the question generation device 100 may be acquired (downloaded) from a predetermined server device or the like via the communication I / F 507.
　補助記憶装置５０８は、例えばＨＤＤやＳＳＤ（Solid State Drive）等であり、プログラムやデータを格納している不揮発性の記憶装置である。補助記憶装置５０８に格納されているプログラムやデータには、例えば、ＯＳ、質問生成装置１００が有する各機能部を実現する１以上のプログラム等がある。 The auxiliary storage device 508 is, for example, an HDD or SSD (Solid State Drive), and is a non-volatile storage device that stores programs and data. The programs and data stored in the auxiliary storage device 508 include, for example, an OS and one or more programs that realize each functional unit included in the question generation device 100.
　本発明の第一の実施形態における質問生成装置１００は、図３に示すハードウェア構成を有することにより、後述する各種処理を実現することができる。なお、図３に示す例では、本発明の第一の実施形態における質問生成装置１００が１台の装置（コンピュータ）で実現される場合について説明したが、これに限られない。本発明の第一の実施形態における質問生成装置１００は、複数台の装置（コンピュータ）で実現されていても良い。また、本発明の第一の実施形態における質問生成装置１００は、複数の演算装置５０６や複数のメモリ（ＲＡＭ５０４、ＲＯＭ５０５、補助記憶装置５０８等）を備える装置（コンピュータ）で実現されていても良い。 The question generation device 100 according to the first embodiment of the present invention has the hardware configuration shown in FIG. In the example illustrated in FIG. 3, the case where the question generation device 100 according to the first embodiment of the present invention is realized by one device (computer) is described, but the present invention is not limited thereto. The question generation device 100 according to the first embodiment of the present invention may be realized by a plurality of devices (computers). Further, the question generation device 100 according to the first embodiment of the present invention may be realized by a device (computer) including a plurality of arithmetic devices 506 and a plurality of memories (RAM 504, ROM 505, auxiliary storage device 508, etc.). .
　（改訂質問の生成処理）
　次に、本発明の第一の実施形態における改訂質問の生成処理について、図４を参照しながら説明する。図４は、本発明の第一の実施形態における改訂質問の生成処理の一例を示すフローチャートである。なお、改訂質問の生成処理では、改訂質問生成部２００を実現する改訂質問生成モデルは学習済みであるものとする。 (Revision question generation process)
Next, the revision question generation process in the first embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing an example of a revision question generation process in the first embodiment of the present invention. In the revised question generation process, it is assumed that the revised question generation model for realizing the revised question generation unit 200 has been learned.
　ここで、本発明の第一の実施形態における改訂質問生成部２００を実現する改訂質問生成モデルの一例を図５に示す。図５に示すように、本発明の第一の実施形態では、改訂質問生成モデルは、Encode Layer、Matching Layer及びDecode Layerの３つ層で構成されるニューラルネットワークである。これらの層のうち、Encode Layer及びMatching Layerによって照合部２１０が実現される。また、Decode Layerによって質問復元部２２０が実現される。以降の改訂質問の生成処理では、図５に示す改訂質問生成モデルを参照も参照しながら、各層の詳細な処理についても説明する。 Here, FIG. 5 shows an example of a revised question generation model for realizing the revised question generation unit 200 in the first embodiment of the present invention. As shown in FIG. 5, in the first embodiment of the present invention, the revised question generation model is a neural network composed of three layers of Encode 、 Layer, Matching 及 び Layer, and Decode Layer. Among these layers, the matching unit 210 is realized by Encode layer and Matching layer. Also, the question restoration unit 220 is realized by Decode layer. In the subsequent revision question generation processing, detailed processing of each layer will be described with reference to the revision question generation model shown in FIG.
　なお、Encode Layer及びDecode Layerは、言語生成のモデルであるSeq2Seqをベースとした層である。一方で、Matching Layerは、機械読解タスクで用いられるAttention Flow Layer及びModeling Layerをベースとした層である。Seq2Seqの詳細については、例えば、以下の参考文献１や参考文献２を参照されたい。また、読解タスクの詳細については、例えば、以下の参考文献３を参照されたい。 The Encode Layer and Decode Layer are layers based on the language generation model Seq2Seq. On the other hand, Matching Layer is a layer based on Attention Flow Layer and Modeling Layer used in machine reading tasks. For details of Seq2Seq, refer to Reference Document 1 and Reference Document 2 below, for example. For details of the reading comprehension task, refer to Reference Document 3 below, for example.
　［参考文献１］
　I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. Proc of the 27th International Conference on Neural Information Processing Systems(NIPS2014), pp. 3104-3112, 2014.
　［参考文献２］
　O. Vinyals and Q. V. Le. A neural conversational model. Proc of the ICML Deep Learning Workshop 2015, 2015.
　［参考文献３］
　M. J. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. Bidirectional attention flow for machine comprehension. Proc of 5th International Conference on Learning Representations(ICLR2017), 2017.
　ステップＳ１０１：改訂質問生成部２００は、質問（入力質問）Ｑと、関連文書Ｘとを入力する。 [Reference 1]
I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks.Proc of the 27th International Conference on Neural Information Processing Systems (NIPS2014), pp. 3104-3112, 2014.
[Reference 2]
O. Vinyals and Q. V. Le. A neural conversational model.Proc of the ICML Deep Learning Workshop 2015, 2015.
[Reference 3]
M. J. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. Bidirectional attention flow for machine comprehension.Proc of 5th International Conference on Learning Representations (ICLR2017), 2017.
Step S101: The revised question generation unit 200 inputs a question (input question) Q and a related document X.
　ステップＳ１０２：改訂質問生成部２００の照合部２１０は、以下のステップＳ１０２－１～ステップＳ１０２－４により、マッチング情報として、Decoderの初期状態とする隠れ状態ベクトルｈｄ０と、機械読解タスクで用いられるマッチングモデルであるマッチング行列Ｍとを生成する。 Step S102: Revised Question matching unit 210 of the generator 200, the following steps S102-1 ~ step S102-4, as the matching information, a hidden state vector h d0 to the initial state of the Decoder, used in the machine reading task A matching matrix M that is a matching model is generated.
　ステップＳ１０２－１：まず、照合部２１０は、図５に示す改訂質問生成モデルのEncode LayerにおけるWord Embeddingの処理として、関連文書Ｘ及び入力質問Ｑをそれぞれｄ次元の単語ベクトル系列に変換する。すなわち、照合部２１０は、関連文書Ｘ及び入力質問Ｑをそれぞれ構成する各単語トークンをベクトル化して単語ベクトル系列を作成する。 Step S102-1: First, the collation unit 210 converts the related document X and the input question Q into d-dimensional word vector sequences as Word Embedding processing in Encode 質問 Layer of the revised question generation model shown in FIG. That is, the matching unit 210 vectorizes each word token constituting the related document X and the input question Q to create a word vector series.
　関連文書Ｘの単語ベクトル系列もＸで表すものとして、関連文書Ｘの単語ベクトル系列Ｘを、 Suppose that the word vector sequence of the related document X is also represented by X, the word vector sequence X of the related document X is
Figure JPOXMLDOC01-appb-M000001
 と表す。
Figure JPOXMLDOC01-appb-M000001
 It expresses.
　また、入力質問Ｑの単語ベクトル系列もＱで表すものとして、入力質問Ｑの単語ベクトル系列Ｑを、 Suppose that the word vector sequence of the input question Q is also represented by Q.
Figure JPOXMLDOC01-appb-M000002
 と表す。
Figure JPOXMLDOC01-appb-M000002
 It expresses.
　なお、本発明の第一の実施形態では、入力された入力質問Ｑ及び関連文書Ｘから単語ベクトル系列Ｘ及びＱを作成したが、これに限られず、例えば、上記のステップＳ１０１で単語ベクトル系列Ｘ及びＱが入力されても良い。 In the first embodiment of the present invention, the word vector series X and Q are generated from the input question Q and the related document X that are input. However, the present invention is not limited to this. For example, in the above step S101, the word vector series X And Q may be input.
　ステップＳ１０２－２：次に、照合部２１０は、図５に示す改訂質問生成モデルのEncode LayerにおけるPassage Contextの処理として、単語ベクトル系列ＸをＲＮＮ（Recurrent Neural Network）によりエンコードして、関連文書Ｘのコンテキスト行列Ｈ∈Ｒ２ｄ×Ｔを得る。なお、コンテキスト行列Ｈの第ｔ列目の要素で構成される列ベクトルをコンテキストベクトルＨｔと表す。 Step S102-2: Next, as a process of Passage Context in the Encode Layer of the revised question generation model shown in FIG. 5, the collation unit 210 encodes the word vector sequence X by RNN (Recurrent Neural Network), and the related document X To obtain a context matrix HεR 2d × T. Note that a column vector composed of elements of the t-th column of the context matrix H is represented as a context vector H t .
　同様に、照合部２１０は、図５に示す改訂質問生成モデルのEncode LayerにおけるQuestion Contextの処理として、単語ベクトル系列ＱをＲＮＮによりエンコードして、入力質問Ｑのコンテキスト行列Ｕ∈Ｒ２ｄ×Ｊを得る。なお、コンテキスト行列Ｕの第ｊ列目の要素で構成される列ベクトルをコンテキストベクトルＵｊと表す。 Similarly, the collation unit 210 encodes the word vector sequence Q by RNN as the Question Context processing in the Encode Layer of the revised question generation model shown in FIG. 5, and sets the context matrix U∈R 2d × J of the input question Q. obtain. Note that a column vector composed of elements in the j-th column of the context matrix U is represented as a context vector U j .
　ここで、Passage Context及びQuestion Contextの処理に用いられるＲＮＮは、例えば、ｂｉ－ＲＮＮ、ＬＳＴＭ（Long Short Term Memory）、ｂｉ－ＬＳＴＭ等であっても良い。ただし、Passage Contextの処理に用いられるＲＮＮと、Question Contextの処理に用いられるＲＮＮとは共通のパラメータを用いる。 Here, the RNN used for the processing of Passage Context and QuestionQuestContext may be, for example, bi-RNN, LSTM (Long Short Term Memory), bi-LSTM, or the like. However, a common parameter is used for the RNN used for the PassagePassContext processing and the RNN used for the Question Context processing.
　ステップＳ１０２－３：次に、照合部２１０は、図５に示す改訂質問生成モデルのMatching Layerの処理として、以下により、Decoderの初期状態とする隠れ状態ベクトルｈｄ０を生成する。 Step S102-3: Next, matching unit 210, as processing of Matching Layer revised question generator model shown in FIG. 5, below, to generate a hidden state vector h d0 to the initial state of the Decoder.
　まず、照合部２１０は、注意機構（attention）を用いて、コンテキストベクトルＵＪ－１及びコンテキスト行列Ｈに対して、以下の式（１）及び式（２）により、関連文書ＸとのアテンションベクトルＨ＾Ｕ∈Ｒ２ｄを計算する。なお、明細書の記載の便宜上、「＾を上に付与したＸ」（すなわち、アクセントとして「＾」が付与されたＸ）を「Ｘ＾」と表記する。 First, the collation unit 210 uses the attention mechanism (attention) to the attention vector U J−1 and the context matrix H for the attention vector with the related document X according to the following equations (1) and (2). to calculate the H ^ U ∈R 2d. For convenience of description, “X with“ ^ ”attached to the top (that is, X with“ ^ ”added as an accent) is expressed as“ X ^ ”.
　ここで、τは転置を表す。また、ｓｏｆｔｍａｘｔはｓｏｆｔｍａｘ関数のｔ番目の出力を表す。なお、上記の式（２）のＨ＾Ｕで下付き表記されている「Ｕ」は添字ではない。 Here, τ represents transposition. Softmax t represents the t-th output of the softmax function. Note that “U” subscripted by H ^ U in the above formula (2) is not a subscript.
　同様に、照合部２１０は、注意機構（attention）を用いて、コンテキストベクトルＵＪ－１及びコンテキスト行列Ｕに対して、以下の式（３）及び式（４）により、入力質問ＱとのアテンションベクトルＵ＾Ｕ∈Ｒ２ｄを計算する。 Similarly, the collation unit 210 uses the attention mechanism (attention) to perform the attention with the input question Q by the following expressions (3) and (4) with respect to the context vector U J−1 and the context matrix U. to calculate the vector U ^ U ∈R 2d.
Figure JPOXMLDOC01-appb-M000004
 　ここで、ｓｏｆｔｍａｘｊはｓｏｆｔｍａｘ関数のｊ番目の出力を表す。なお、上記の式（４）のＵ＾Ｕで下付き表記されている「Ｕ」は添字ではない。
Figure JPOXMLDOC01-appb-M000004
 Here, softmax j represents the jth output of the softmax function. Note that “U” subscripted by U ^ U in the above formula (4) is not a subscript.
　これは、入力質問Ｑのコンテキスト自身でアテンションを取ることになり、入力質問Ｑ中の重要な単語を考慮するためのものである。 This is because the context itself of the input question Q takes attention, and important words in the input question Q are taken into consideration.
　そして、照合部２１０は、上記の式（２）及び（４）でそれぞれ計算した２つのアテンションベクトルＨ＾Ｕ及びＵ＾Ｕを用いて、以下の式（５）により、Decoderの初期状態とする隠れ状態ベクトルｈｄ０を計算する。 Then, the matching unit 210 sets the initial state of the Decoder according to the following equation (5) using the two attention vectors H ^ U and U ^ U calculated in the above equations (2) and (4), respectively. The hidden state vector hd0 is calculated.
Figure JPOXMLDOC01-appb-M000005
 　ここで、Ｗｍ∈Ｒ４ｄ×２ｄ及びｂｍ∈Ｒ２ｄはパラメータである。また、ｆは活性化関数であり、例えば、Leaky ReLU等を用いる。なお、［；］は連結を表す。
Figure JPOXMLDOC01-appb-M000005
 Here, W m εR 4d × 2d and b m εR 2d are parameters. F is an activation function, and for example, Leaky ReLU or the like is used. In addition, [;] represents a connection.
　ステップＳ１０２－４：次に、照合部２１０は、図５に示す改訂質問生成モデルのMatching Layerの処理として、以下により、マッチング行列Ｍを生成する。 Step S102-4: Next, the collation unit 210 generates a matching matrix M as follows as the Matching layer processing of the revised question generation model shown in FIG.
　まず、照合部２１０は、系列長がＴであるコンテキスト行列Ｈと、系列長がＪであるコンテキスト行列ＵとをAttention層に入力する。そして、照合部２１０は、Attention層の処理として、関連文書Ｘと入力質問Ｑとの単語の類似度行列Ｓを計算する。 First, the collation unit 210 inputs a context matrix H having a sequence length of T and a context matrix U having a sequence length of J to the attention layer. And the collation part 210 calculates the similarity matrix S of the word of the related document X and the input question Q as a process of Attention layer.
　関連文書Ｘのｔ番目の単語と、入力質問Ｑのｊ番目の単語との類似度を、 The similarity between the t-th word of the related document X and the j-th word of the input question Q is
Figure JPOXMLDOC01-appb-M000006
 と定義する。ここで、ｗｓ τ∈Ｒ６ｄはパラメータである。また、
Figure JPOXMLDOC01-appb-M000006
 It is defined as Here, w s τ ∈R 6d it is a parameter. Also,
Figure JPOXMLDOC01-appb-M000007
 は要素積を表す。
Figure JPOXMLDOC01-appb-M000007
 Represents an element product.
　これにより、類似度行列Ｓ＝（Ｓｔｊ）∈ＲＴ×Ｊが作成される。 Thereby, a similarity matrix S = (S tj ) εR T × J is created.
　次に、照合部２１０は、類似度行列Ｓを用いて、関連文書Ｘから入力質問Ｑへのアテンションと、入力質問Ｑから関連文書Ｘへのアテンションとの２方向のアテンションを計算する。 Next, using the similarity matrix S, the collation unit 210 calculates attentions in two directions, an attention from the related document X to the input question Q and an attention from the input question Q to the related document X.
　関連文書Ｘから入力質問Ｑへのアテンションでは、照合部２１０は、関連文書Ｘの各単語について、入力質問Ｑの単語で重み付けしたアテンションベクトルを計算する。すなわち、照合部２１０は、以下の式（７）及び（８）により、関連文書Ｘのｔ番目の単語に対応するアテンションベクトル In the attention from the related document X to the input question Q, the collation unit 210 calculates an attention vector weighted by the word of the input question Q for each word of the related document X. That is, the collation unit 210 uses the following equations (7) and (8) to indicate the attention vector corresponding to the t-th word of the related document X.
Figure JPOXMLDOC01-appb-M000008
 を計算する。
Figure JPOXMLDOC01-appb-M000008
 Calculate
Figure JPOXMLDOC01-appb-M000009
 　また、入力質問Ｑから関連文書Ｘへのアテンションでは、照合部２１０は、入力質問Ｑのいずれかの単語に強く関連する単語で重み付けしたアテンションベクトルを計算した上で、このアテンションベクトルを関連文書Ｘの系列長Ｔ分並べた行列を作成する。すなわち、まず、照合部２１０は、以下の式（９）及び（１０）により、アテンションベクトル
Figure JPOXMLDOC01-appb-M000009
 Further, in the attention from the input question Q to the related document X, the collation unit 210 calculates an attention vector weighted by a word strongly related to any word of the input question Q, and then uses this attention vector as the related document X. A matrix arranged for the sequence length T is created. That is, first, the matching unit 210 uses the following equations (9) and (10) to obtain an attention vector.
Figure JPOXMLDOC01-appb-M000010
 を計算する。
Figure JPOXMLDOC01-appb-M000010
 Calculate
Figure JPOXMLDOC01-appb-M000011
 　ここで、ｍａｘｊ（Ｓ）は、ｔ＝１，・・・，Ｔ－１に対して、ｍａｘ（Ｓｔ）となる、ベクトルＳｔのｊ番目の要素Ｓｔｊを要素とするＴ次元のベクトルである（なお、各γｔを要素とするベクトルγはＴ次元のベクトルである。）。
Figure JPOXMLDOC01-appb-M000011
 Here, max j (S) is, t = 1, ···, against T-1, the max (S t), the T dimension of the j-th element S tj the elements of the vector S t It is a vector ( note that a vector γ having each γ t as an element is a T-dimensional vector).
　続いて、照合部２１０は、上記の式（１０）で計算されたアテンションベクトルをＴ個並べた行列 Subsequently, the matching unit 210 is a matrix in which T attention vectors calculated by the above equation (10) are arranged.
Figure JPOXMLDOC01-appb-M000012
 を作成する。
Figure JPOXMLDOC01-appb-M000012
 Create
　その後、照合部２１０は、コンテキストベクトルＨＴ－１とコンテキスト行列Ｈとのself-attentionをとったアテンションベクトルＨ＾Ｈ∈Ｒ２ｄ×Ｔを用いて、以下の式（１１）により、アテンション行列Ｇを計算する。 After that, the matching unit 210 uses the attention vector H ^ H ∈ R 2d × T obtained by taking the self-attention between the context vector H T-1 and the context matrix H, by the following equation (11), and the attention matrix G Calculate
Figure JPOXMLDOC01-appb-M000013
 　なお、self-attentionの詳細については、例えば、以下の参考文献４を参照されたい。
Figure JPOXMLDOC01-appb-M000013
 For details of self-attention, refer to Reference Document 4 below, for example.
　［参考文献４］
　W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou. Gated self-matching networks for reading comprehension and question answering. Proc of the 55th Annual Meeting of the Association for Computational Linguistics (ACL2017), pp.189-198, 2017.
　ただし、照合部２１０は、アテンションベクトルＨ＾Ｈ∈Ｒ２ｄを用いずに（すなわち、上記の式（１１）でアテンションベクトルＨ＾Ｈを連結せずに）、アテンション行列Ｇを計算しても良い。この場合、アテンション行列Ｇは、Ｇ∈Ｒ８ｄ×Ｔとなる。 [Reference 4]
W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou.Gated self-matching networks for reading comprehension and question answering.Proc of the 55th Annual Meeting of the Association for Computational Linguistics (ACL2017), pp .189-198, 2017.
However, the matching unit 210 may calculate the attention matrix G without using the attention vector H ^ H ∈ R 2d (that is, without concatenating the attention vector H ^ H in the above equation (11)). . In this case, the attention matrix G is GεR 8d × T.
　そして、照合部２１０は、図５に示す改訂質問生成モデルのEncode LayerにおけるMatching Modelの処理として、上記の式（１１）で計算されたアテンション行列ＧをＲＮＮに入力してマッチング行列Ｍ∈Ｒ２ｄ×Ｔを得る。 Then, the matching unit 210, as processing of Matching Model in Encode Layer revised question generator model shown in FIG. 5, the matching by entering the attention matrix G calculated by the above equation (11) to the RNN matrix M∈R 2d XT is obtained.
　以上のステップＳ１０２により、マッチング情報として、Decoderの初期状態とする隠れ状態ベクトルｈｄ０と、機械読解タスクで用いられるマッチングモデルであるマッチング行列Ｍとが生成される。 In step S102 described above, as the matching information, a hidden state vector h d0 to the initial state of the Decoder, is generated and match matrix M is a matching model used in the machine reading task.
　なお、マッチング情報を生成する方法として、上記以外の任意の方法を用いても良い。また、マッチング情報の表現形式として、ベクトルや行列、テンソル等の任意の形式が用いられても良い。例えば、入力質問Ｑと関連文書Ｘとで一致した単語の要素を１、それ以外の単語の要素を０としたbag-of-wordsベクトルを用いても良いし、単語の種類の一致だけでなく、関連文書Ｘ中の単語の出現位置まで考慮した情報を用いても良い。ただし、マッチング情報が類似度等のスカラー値のみで表現される場合には入力質問Ｑと関連文書Ｘとがどの部分で一致しているかの情報が欠落してしまうため、マッチング情報の表現形式としてはスカラー値でないことが好ましい。 In addition, you may use arbitrary methods other than the above as a method of producing | generating matching information. In addition, any format such as a vector, a matrix, or a tensor may be used as an expression format of the matching information. For example, it is possible to use a bag-of-words vector in which the word element matched between the input question Q and the related document X is 1, and the other word elements are 0. Information that takes into account the appearance position of the word in the related document X may be used. However, when the matching information is expressed only by a scalar value such as a similarity, information on which part the input question Q and the related document X match is lost. Is preferably not a scalar value.
　ステップＳ１０３：改訂質問生成部２００の質問復元部２２０は、照合部２１０が生成したマッチング情報（隠れ状態ベクトルｈｄ０及びマッチング行列Ｍ）と、入力質問Ｑと、関連文書Ｘとを用いて、以下のステップＳ１０３－１～ステップＳ１０３－７により、改訂質問ＲＱとなる自然文を生成する。 Step S103: The question restoration unit 220 of the revised question generation unit 200 uses the matching information (the hidden state vector hd0 and the matching matrix M) generated by the matching unit 210, the input question Q, and the related document X to be described below. Steps S103-1 to S103-7 generate a natural sentence that becomes the revised question RQ.
　ここで、改訂質問ＲＱとなる自然文は、単語ｙｓ（ｓ＝０，１，・・・）により構成されているものとする。ただし、単語ｙ０は、文の始端を示すトークン＜ＢＯＳ＞であるものとする。質問復元部２２０は、例えば、文の終端を示すトークン＜ＥＯＳ＞が生成されるまで、ｓ＝１から順に、単語ｙｓを繰り返し生成することで、改訂質問ＲＱを生成する。以下のステップＳ１０３－１～ステップＳ１０３－７では、或るｓにおける単語ｙｓを生成する場合について説明する。また、DecoderであるＲＮＮはＬＳＴＭであるものとして、このＬＳＴＭの隠れ状態をｈｄｓと表し、この隠れ状態の初期値（すなわち、ｓ＝０である場合の隠れ状態ｈｄｓ）を、照合部２１０で計算された隠れ状態ベクトルｈｄ０とする。 Here, the natural sentence that becomes the revised question RQ is assumed to be composed of the word y s (s = 0, 1,...). However, the word y 0 is assumed to be a token that indicates the beginning of a sentence <BOS>. Question restoring unit 220, for example, until the token <EOS> indicating the end of a sentence is generated, from s = 1 in the order, by repeatedly generating word y s, and generates a revised question RQ. In the following steps S103-1 ~ step S103-7, will be described for generating a word y s at a certain s. Further, assuming that the RNN that is the Decoder is LSTM, the hidden state of this LSTM is represented as h ds, and the initial value of this hidden state (that is, the hidden state h ds when s = 0) is referred to as the collating unit 210 The hidden state vector hd0 calculated in (1) is used.
　ステップＳ１０３－１：まず、質問復元部２２０は、図５に示す改訂質問生成モデルのDecode LayerにおけるWord Embeddingの処理として、１つ前の繰り返しで生成された単語ｙｓ－１を単語ベクトルｅｙｓ－１に変換する。なお、上述したように、ｓ＝１である場合（すなわち、初回である場合）、単語ｙｓ－１＝ｙ０として、文の始端を示すトークン＜ＢＯＳ＞が単語ベクトルｅｙ０に変換される。 Step S103-1: First, as the Word Embedding process in the Decode Layer of the revised question generation model shown in FIG. 5, the question restoration unit 220 uses the word y s-1 generated in the previous iteration as the word vector e ys. Convert to -1 . As described above, when s = 1 (that is, for the first time), the token <BOS> indicating the beginning of the sentence is converted into the word vector e y0 with the word y s−1 = y 0. .
　ステップＳ１０３－２：次に、質問復元部２２０は、図５に示す改訂質問生成モデルのDecode Layerの処理として、注意機構（attention）を用いて、以下の式（１２）～（１５）により、DecoderであるＬＳＴＭへの入力ｚ＾ｓ∈Ｒ３ｄを計算する。 Step S103-2: Next, the question restoration unit 220 uses the attention mechanism (attention) as the processing of the decode layer of the revised question generation model shown in FIG. 5 according to the following equations (12) to (15). Calculate the input z ^ s R 3d to the LSTM which is the Decoder.
Figure JPOXMLDOC01-appb-M000014
 　ここで、Ｗｄ∈Ｒ２ｄ×３ｄ及びｂｄ∈Ｒ２ｄはパラメータ、ｆは活性化関数である。また、Ｍｔ∈Ｒ２ｄはマッチング行列Ｍの第ｔ列目の要素で構成される列ベクトルである。
Figure JPOXMLDOC01-appb-M000014
 Here, W d εR 2d × 3d and b d εR 2d are parameters, and f is an activation function. M t εR 2d is a column vector composed of elements of the t-th column of the matching matrix M.
　ステップＳ１０３－３：次に、質問復元部２２０は、Decoderの隠れ状態ｈｄｓを以下の式（１６）により更新する。 Step S103-3: Next, the question restoration unit 220 updates the hidden state h ds of the Decoder according to the following equation (16).
Figure JPOXMLDOC01-appb-M000015
 　ステップＳ１０３－４：次に、質問復元部２２０は、Decode LayerにおけるDecoderの処理として、上記の式（１５）で得られたｚ＾ｓをＬＳＴＭに入力して、softmax関数を計算する。これにより、softmax関数の出力として、生成確率分布ＰＧ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）が得られる。生成確率分布ＰＧ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）は、ｓ－１番目までの単語ｙｓが生成された場合に、ｓ番目の単語ｙｓとして、予め設定された或る特定の語彙集合に含まれる単語が生成される条件付き確率の分布である。なお、特定の語彙集合としては、例えば、一般的な文書に頻出する単語によって構成される集合等が挙げられる。
Figure JPOXMLDOC01-appb-M000015
 Step S103-4: Next, the question restoration unit 220 inputs z ^ s obtained by the above equation (15) to the LSTM as a Decoder process in the Decode Layer, and calculates a softmax function. As a result, a generation probability distribution P G (y s | y <s , X, Q) is obtained as an output of the softmax function. The generation probability distribution P G (y s | y <s , X, Q) is a specific identification that is set in advance as the s-th word y s when the s−1th words y s are generated. This is a distribution of conditional probabilities that a word included in the vocabulary set is generated. The specific vocabulary set includes, for example, a set composed of words that appear frequently in a general document.
　ステップＳ１０３－５：次に、質問復元部２２０は、Decode Layerにおける処理として、上記の式（１３）で得られた重みεｓｔと、softmax関数とを用いて、以下の式（１７）により、生成確率ＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）を計算する。 Step S103-5: Next, the question restoration unit 220 uses the weight ε st obtained in the above equation (13) and the softmax function as processing in the Decode Layer, according to the following equation (17): The generation probability P C (y s | y <s , X, Q) is calculated.
Figure JPOXMLDOC01-appb-M000016
 　ここで、Ｉ（ｙｓ＝ｘｔ）は、生成する単語ｙｓが関連文書Ｘのｔ番目の単語ｘｔと一致する場合は１、それ以外の場合は０を返す関数である。
Figure JPOXMLDOC01-appb-M000016
 Here, I (y s = x t ) is a function that returns 1 if the generated word y s matches the t-th word x t of the related document X, and returns 0 otherwise.
　上記の生成確率ＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）は、ＣｏｐｙＮｅｔの考え方を応用したものである。ＣｏｐｙＮｅｔとは、単語の生成確率をＬＳＴＭの出力の外からも与えることで、エンコード側の単語をそのまま生成（コピー）し易くするニューラルネットワークモデルである。本発明の第一の実施形態では、この生成確率ＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）を導入することで、ｓ番目の単語ｙｓとして、関連文書Ｘに含まれる単語が生成（コピー）され易くすることができる。したがって、ＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）を導入することで、欠損質問と見做された入力質問Ｑを、関連文書Ｘに含まれる単語で補うことができるようになる。なお、ＣｏｐｙＮｅｔの詳細は、例えば、以下の参考文献５や参考文献６を参照されたい。 The generation probability P C (y s | y <s , X, Q) is an application of the concept of CopyNet. CopyNet is a neural network model that facilitates generating (copying) an encoded word as it is by giving the word generation probability from outside the LSTM output. In the first embodiment of the present invention, a word included in the related document X is generated as the sth word y s by introducing the generation probability P C (y s | y <s , X, Q). (Copying) can be facilitated. Therefore, by introducing P C (y s | y <s , X, Q), the input question Q that is regarded as a missing question can be supplemented with the words included in the related document X. For details of CopyNet, refer to Reference Document 5 and Reference Document 6 below, for example.
　［参考文献５］
　Z. Cao, C. Luo, W. Li, and S. Li. Joint copying and restricted generation for paraphrase. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence(AAAI2017), pp. 3152-3158, 2017.
　［参考文献６］
　J. Gu, Z. Lu, H. Li, and V. O. Li. Incorporating copying mechanism in sequence-to-sequence learning. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL2016), pp. 1631-1640, 2016.
　ステップＳ１０３－６：次に、質問復元部２２０は、重みλｓを用いて、単語ｙｓの最終的な生成確率Ｐ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）を以下の式（１８）により計算する。 [Reference 5]
Z. Cao, C. Luo, W. Li, and S. Li.Joint copying and restricted generation for paraphrase.Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI2017), pp. 3152-3158, 2017.
[Reference 6]
J. Gu, Z. Lu, H. Li, and VO Li.Incorporating copying mechanism in sequence-to-sequence learning.Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL2016), pp. 1631-1640, 2016 .
Step S103-6: Next question restoration unit 220 uses the weighting lambda s, the final generation probability P of a word y s (y s | y < s, X, Q) following equation (18) Calculate with
Figure JPOXMLDOC01-appb-M000017
 　ここで、重みλｓは、以下の式（１９）により計算される。
Figure JPOXMLDOC01-appb-M000017
 Here, the weight λ s is calculated by the following equation (19).
Figure JPOXMLDOC01-appb-M000018
 　ここで、Ｗλ∈Ｒ１×２ｄ及びｂλ∈Ｒ１はパラメータ、σはシグモイド関数を表す。
Figure JPOXMLDOC01-appb-M000018
 Here, W λ ∈ R 1 × 2d and b λ ∈ R 1 are parameters, and σ is a sigmoid function.
　上記の生成確率Ｐ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）は、重みλｓによるＰＧ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）とＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）との加重平均である。このため、重みλｓによって、関連文書Ｘに含まれる単語がｙｓとしてコピーされるか否かが決定される。 The generation probabilities P (y s | y <s , X, Q) are expressed as P G (y s | y <s , X, Q) and P C (y s | y <s , X, Q) with the weight λ s . Q) is a weighted average. For this reason, whether or not the word included in the related document X is copied as y s is determined by the weight λ s .
　ステップＳ１０３－７：次に、質問復元部２２０は、上記の式（１８）で計算された最終的な生成確率Ｐ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）により単語ｙｓを生成する。すなわち、質問復元部２２０は、例えば、関連文書Ｘ及び入力質問Ｑに含まれる各単語のうち、Ｐ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）が最大となる単語をｙｓとして生成する。 Step S103-7: Next, the question restoration unit 220 generates a word y s based on the final generation probability P (y s | y <s , X, Q) calculated by the above equation (18). That is, for example, the question restoration unit 220 generates, as y s , the word having the maximum P (y s | y <s , X, Q) among the words included in the related document X and the input question Q.
　以上のステップＳ１０３－１～ステップＳ１０３－７を、単語ｙｓとして＜ＥＯＳ＞が生成されるまで繰り返すことで、各単語ｙｓ（ｓ＝０，１，・・・）により構成される改訂質問ＲＱが生成される。この改訂質問ＲＱは、改訂質問生成部２００により、所定の出力先に出力される。ここで、所定の出力先としては、例えば、表示装置５０２や補助記憶装置５０８、他のプログラム（例えば、質問応答タスクを実行するプログラム）等が挙げられる。 Step S103-1 ~ Step S103-7 above, by repeated until the word y s is <EOS> is generated, revised question constituted by each word y s (s = 0,1, ··· ) RQ is generated. The revised question RQ is output by the revised question generation unit 200 to a predetermined output destination. Here, examples of the predetermined output destination include the display device 502, the auxiliary storage device 508, and other programs (for example, a program for executing a question answering task).
　ここで、改訂質問ＲＱは、入力質問Ｑをベースに関連情報Ｘ内の情報を付与することで作成される。このとき、マッチング情報のみを用いて、Encoder-Decoderモデル等のような生成モデルによって改訂質問ＲＱを生成した場合、関連文書Ｘや入力質問Ｑとあまり関係のない改訂質問ＲＱが生成されてしまうことがある。そこで、本発明の第一の実施形態では、ＣｏｐｙＮｅｔの考え方を応用した手法により、マッチング情報だけでなく、関連文書Ｘ自体の情報も利用することで、欠損質問と見做した入力質問Ｑに対して、関連文書Ｘに関係のある改訂質問ＲＱを生成することができるようになる。 Here, the revised question RQ is created by adding information in the related information X based on the input question Q. At this time, when the revised question RQ is generated by a generation model such as an Encoder-Decoder model using only matching information, a revised question RQ that is not very related to the related document X or the input question Q is generated. There is. Therefore, in the first embodiment of the present invention, by using not only the matching information but also the information of the related document X itself by a method applying the concept of CopyNet, the input question Q that is regarded as a missing question is used. Thus, the revised question RQ related to the related document X can be generated.
　なお、上記のステップＳ１０３－７では、各ｓに対して１つの単語ｙｓを生成したが、これに限られず、或るｓ（又は全てのｓ）に対して複数の単語ｙｓを生成しても良い。複数の単語ｙｓを生成することで、例えば、ビームサーチ等を用いて、複数の改訂質問ＲＱを生成することができるようになる。ビームサーチとは、グラフの幅優先探索のような探索アルゴリズムの一種である。ビームサーチを用いる場合、質問復元部２２０は、例えば、各ｓに対して、Ｂ個のビーム幅分の単語ｙｓを生成する。これにより、最終的に生成された改訂質問ＲＱの単語長をＬとした場合、ＢＬ個の改訂質問ＲＱの候補が生成される。次に、質問復元部２２０は、これらの候補から、ビームサーチを用いて、生成スコア順に並べて上位ｑ個を出力することで、複数のバリエーションの改訂質問ＲＱを生成することができる。 In step S103-7, one word y s is generated for each s. However, the present invention is not limited to this, and a plurality of words y s are generated for a certain s (or all s). May be. By generating a plurality of words y s, for example, by using a beam search or the like, it is possible to generate a plurality of revised question RQ. A beam search is a kind of search algorithm such as a breadth-first search of a graph. When using a beam search, the question restoration unit 220 generates, for example, a word y s for B beam widths for each s. Thus, if the word length of the finally produced, revised question RQ is L, candidate B L number of revised question RQ is generated. Next, the question restoration unit 220 can generate revised questions RQ of a plurality of variations by outputting the top q items from these candidates in the order of generation score using a beam search.
　また、上記のステップＳ１０３－１～ステップＳ１０３－７では、単語ｙ０を＜ＢＯＳ＞として、文頭の単語から順に改訂質問ＲＱを生成する場合について説明したが、これに限られず、例えば、単語ｙ０を＜ＥＯＳ＞として、文末の単語から順に改訂質問ＲＱを生成しても良い。 Further, in the above step S103-1 ~ step S103-7, the words y 0 as <BOS>, a case has been described in which the words in the beginning of a sentence to produce a revised question RQ in the order, not limited to this, for example, a word y The revision question RQ may be generated in order from the word at the end of the sentence, with 0 as <EOS>.
　（部分生成及び全体生成）
　ここで、本発明の第一の実施形態における改訂質問の生成処理では、欠損質問と見做された入力質問Ｑの一部の欠損を補った改訂質問ＲＱが生成されても良いし、入力質問Ｑの全ての欠損を補った改訂質問ＲＱが生成されても良い。以降では、入力質問Ｑの一部の欠損を補った改訂質問ＲＱを生成することを「部分生成」、入力質問Ｑの全ての欠損を補った改訂質問ＲＱを生成することを「全体生成」と表す。 (Partial generation and total generation)
Here, in the revised question generation process according to the first embodiment of the present invention, a revised question RQ that compensates for a partial deficiency of the input question Q regarded as a missing question may be generated. A revised question RQ that compensates for all deficiencies in Q may be generated. In the following, generating a revised question RQ that compensates for some deficiencies in the input question Q is “partial generation”, and generating a revised question RQ that compensates for all deficiencies in the input question Q is “general generation”. Represent.
　具体的には、例えば、質問内容が明確であり、かつ、回答に必要な情報の不足がない質問（以降では、このような質問を「全体質問」と表す。）を「プランＡを途中解約するときの料金は？」であり、入力質問Ｑを「料金は？」であるとする。 Specifically, for example, a question with clear contents of a question and lack of information necessary for answering (hereinafter, such a question is referred to as “whole question”) is canceled as “plan A is canceled halfway. "What is the charge when doing?" And the input question Q is "What is the charge?"
　この場合、部分生成では、例えば、改訂質問ＲＱとして「途中解約するときの料金は？」が生成される。一方で、全体生成では、例えば、改訂質問ＲＱとして全体質問「プランＡを途中解約するときの料金は？」が生成される。 In this case, in the partial generation, for example, “What is the charge for canceling midway?” Is generated as the revised question RQ. On the other hand, in the whole generation, for example, the whole question “What is the fee for canceling plan A halfway?” Is generated as the revised question RQ.
　したがって、この場合、部分生成によって全体質問を得るためには、改訂質問ＲＱとして得られた「途中解約するときの料金は？」を入力質問Ｑとして、再度、改訂質問の生成処理を行う必要がある。これより、最終的な改訂質問ＲＱとして全体質問「プランＡを途中解約するときの料金は？」が得られる。 Therefore, in this case, in order to obtain the entire question by partial generation, it is necessary to perform the revised question generation process again using the input question Q as “What is the fee for cancellation on the way?” Obtained as the revised question RQ. is there. As a result, the overall question “What is the fee for canceling plan A halfway?” Is obtained as the final revised question RQ.
　上述したように、部分生成を用いる場合、全体質問を得るためには改訂質問の生成処理を繰り返し実行する必要があるが、一般に、全体生成よりも部分生成の方が、全体質問を高い精度で復元することができる。 As described above, when partial generation is used, it is necessary to repeatedly execute the revision question generation process in order to obtain the entire question, but in general, partial generation is more accurately performed than partial generation. Can be restored.
　なお、改訂質問の生成処理が部分生成であるか又は全体生成であるかは、改訂質問生成モデルの学習処理に用いられる学習データセットによって決定される。また、改訂質問の生成処理を部分生成とするか又は全体生成とするかは、改訂質問が用いられる質問応答タスクに応じて決定される。 It should be noted that whether the revision question generation process is partial generation or total generation is determined by a learning data set used for the revision question generation model learning process. Whether the revision question generation process is partial generation or total generation is determined according to a question answering task in which the revision question is used.
　ここで、学習データセットとは、正解データとして用いる入力質問Ｑと、関連文書Ｘとの組で表される学習データの集合である。また、正解データとして用いる入力質問Ｑを構成する各単語に対して、該単語が関連文書Ｘに含まれる単語である場合は１、それ以外の場合は０となるラベルが付与されているものとする。以降では、便宜上、正解データとして用いる入力質問Ｑを「正解質問Ｑｔｒｕｅ」と表す。 Here, the learning data set is a set of learning data represented by a set of the input question Q used as correct answer data and the related document X. In addition, for each word constituting the input question Q used as correct answer data, a label that is 1 if the word is a word included in the related document X, and 0 otherwise is given. To do. Hereinafter, for the sake of convenience, the input question Q used as correct answer data is represented as “correct answer question Q true ”.
　（改訂質問生成モデルの学習処理）
　次に、本発明の第一の実施形態における改訂質問生成モデルの学習処理について、図６を参照しながら説明する。図６は、本発明の第一の実施形態における改訂質問生成モデルの学習処理の一例を示すフローチャートである。なお、改訂質問生成モデルの学習処理は、例えば、学習データセットを所定の個数のミニバッチに分割した上で、ミニバッチ毎に、改訂質問生成モデルのパラメータを更新する。 (Learning process of revised question generation model)
Next, the learning process of the revised question generation model in the first embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart showing an example of the learning process of the revised question generation model in the first embodiment of the present invention. In the learning process of the revised question generation model, for example, the learning data set is divided into a predetermined number of mini-batches, and the parameters of the revised question generation model are updated for each mini-batch.
　以下のステップＳ２０１～ステップＳ２０４は、ミニバッチに含まれる各学習データを用いて繰り返し実行される。一方で、以下のステップＳ２０５～ステップＳ２０６は、ミニバッチに含まれる全ての学習データに対してステップ２０１～ステップＳ２０４が実行された後に実行される。 The following steps S201 to S204 are repeatedly executed using each learning data included in the mini-batch. On the other hand, the following steps S205 to S206 are executed after steps 201 to S204 are executed for all learning data included in the mini-batch.
　ステップＳ２０１：欠損質問作成部３００は、学習データに含まれる正解質問Ｑｔｒｕｅを入力する。また、改訂質問生成モデル学習部４００は、学習データに含まれる正解質問Ｑｔｒｕｅ及び関連文書Ｘを入力する。 Step S201: The missing question creation unit 300 inputs a correct answer question Q true included in the learning data. Further, the revised question generation model learning unit 400 inputs the correct answer question Q true and the related document X included in the learning data.
　ステップＳ２０２：次に、欠損質問作成部３００は、正解質問Ｑｔｒｕｅの一部を欠損させた質問Ｑ（欠損質問Ｑ）を作成する。ここで、正解質問Ｑｔｒｕｅに対する欠損質問Ｑのバリエーションは一般に複数存在するが、欠損質問作成部３００は、これらの全ての欠損質問Ｑを作成しても良いし、一部（１つも含む）の欠損質問Ｑを作成しても良い。 Step S202: Next, the missing question creation unit 300 creates a question Q (missing question Q) in which a part of the correct question Q true is missing. Here, there are generally a plurality of variations of the missing question Q with respect to the correct answer question Q true . However, the missing question creation unit 300 may create all of these missing questions Q, or some (including one) of them. A missing question Q may be created.
　例えば、正解質問Ｑｔｒｕｅが「プランＡの料金を教えて」であるとする。この場合、欠損質問Ｑのバリエーションとして、「料金を教えて」、「教えて」が存在する。したがって、欠損質問作成部３００は、「料金を教えて」と「教えて」との両方の欠損質問Ｑを作成しても良いし、「料金を教えて」又は「教えて」のいずれかの欠損質問Ｑを作成しても良い。 For example, it is assumed that the correct answer question Q true is “tell me the fee for plan A”. In this case, “Tell me a fee” and “Tell me” exist as variations of the missing question Q. Therefore, the missing question creation unit 300 may create the missing question Q for both “tell me the fee” and “tell me”, and either “tell me the fee” or “tell me” A missing question Q may be created.
　なお、部分生成を実現する改訂質問生成モデルを学習する場合、正解質問Ｑｔｒｕｅと同一の全体質問文を欠損質問Ｑとした上で、正解質問Ｑｔｒｕｅとして文頭を示すトークン＜ＢＯＳ＞を設定しても良い。これより、例えば、部分生成による改訂質問の生成処理を行う場合に、単語ｙ１として＜ＢＯＳ＞が生成されたとき、改訂質問ＲＱとして全体質問が生成されたことを知ることができる。 When learning a revised question generation model that realizes partial generation, the entire question sentence that is the same as the correct question Q true is set as a missing question Q, and a token <BOS> that indicates the beginning of the sentence is set as the correct question Q true. May be. From this, for example, when performing processing for generating revised question by partial generation, when <BOS> is generated as a word y 1, it can be seen that the entire question as revised question RQ is generated.
　例えば、全体質問を「プランＡを途中解約するときの料金は？」であるとする。この場合、１回目の部分生成では、入力質問Ｑ「料金は？」から改訂質問ＲＱ「途中解約するときの料金は？」が生成される。次に、２回目の部分生成では、入力質問Ｑ「途中解約するときの料金は？」から改訂質問ＲＱ「プランＡを途中解約するときの料金は？」が生成される。そして、３回目の部分生成では、入力質問Ｑ「プランＡを途中解約するときの料金は？」から改訂質問ＲＱ「＜ＢＯＳ＞」が生成される。＜ＢＯＳ＞が生成されるということは、これ以上追加（生成）可能な文節が存在しないことを示す。このため、２回目の改訂質問ＲＱ「プランＡを途中解約するときの料金は？」が全体質問であると知ることができる。 Suppose, for example, that the overall question is "What is the fee for canceling Plan A halfway?" In this case, in the first partial generation, the revised question RQ “What is the fee for canceling midway?” Is generated from the input question Q “What is the fee?”. Next, in the second partial generation, the revised question RQ “What is the fee for canceling plan A halfway?” Is generated from the input question Q “What is the fee for canceling midway?”. Then, in the third partial generation, the revised question RQ “<BOS>” is generated from the input question Q “What is the fee for canceling plan A halfway?”. Generation of <BOS> indicates that there is no more clause that can be added (generated). For this reason, it can be known that the second revised question RQ “What is the fee for canceling Plan A halfway?” Is the entire question.
　ここで、欠損質問Ｑの作成方法は任意の方法を用いることができるが、欠損質問Ｑの作成方法としては、例えば、正解質問Ｑｔｒｕｅの係り受け解析や句構造解析等の構文解析を行った結果を用いて作成することができる。また、正解質問Ｑｔｕｒｅから欠損させる部分の粒度も任意に設定することができる。 Here, any method can be used as a method for creating the missing question Q. As a method for creating the missing question Q, for example, syntactic analysis such as dependency analysis of the correct answer Q true or phrase structure analysis was performed. Can be created using the results. In addition, the granularity of the portion to be deleted from the correct answer question Q true can be set arbitrarily.
　欠損質問Ｑの作成方法の一例として、例えば、先頭から文節を順に欠損させる方法が挙げられる。例えば、正解質問Ｑｔｒｕｅが「プランＡを途中解約するときの料金は？」であったとする。この正解質問Ｑｔｒｕｅは、「プランＡを」と「途中解約するときの」と「料金は？」との３文節で構成されている。このため、この場合、欠損質問作成部３００は、例えば、正解質問Ｑｔｒｕｅの先頭の１文節を欠損させた「途中解約するときの料金は？」と、正解質問Ｑｔｒｕｅの先頭の２文節を欠損させた「料金は？」とを欠損質問Ｑとして作成する。 As an example of a method for creating the missing question Q, for example, there is a method of sequentially deleting phrases from the top. For example, it is assumed that the correct answer question Q true is “What is the charge for canceling Plan A halfway?”. This correct answer question Q true is composed of three clauses of “plan A”, “when canceling halfway”, and “how much do you charge?”. For this reason, in this case, the missing question creating section 300 is, for example, "the fee at the time of closeout?" That was missing one clause of the beginning of the correct question Q true and, two clauses of the beginning of the correct question Q true Create the missing question Q as “What is the charge?”
　また、欠損質問Ｑの作成方法の他の例として、例えば、正解質問Ｑｔｒｕｅから係り受け関係にある任意の２文節を抽出して、抽出した２文節を係り受け関係通りに結合した文を欠損質問Ｑとする方法が挙げられる。このとき、得られた欠損質問Ｑと係り受け関係にある文節が正解質問Ｑｔｒｕｅに存在する場合は、更に、当該欠損質問Ｑと当該文節とを結合した文を新たな欠損質問Ｑとしても良い。 As another example of a method for creating a missing question Q, for example, an arbitrary two clauses having a dependency relationship are extracted from a correct question Q true , and a sentence obtained by combining the extracted two clauses according to the dependency relationship is lost. There is a method of using the question Q. At this time, if there is a clause having a dependency relationship with the obtained missing question Q in the correct question Q true , a sentence obtained by combining the missing question Q and the clause may be used as a new missing question Q. .
　また、正解質問Ｑｔｒｕｅが英語等の言語である場合には、句構造解析や係り受け木の解析等を行い、この解析結果から節又は単語単位での欠損を行うことで欠損質問Ｑを作成すれば良い。例えば、正解質問Ｑｔｒｕｅが英語である場合、名詞句（ＮＰ：noun phrase）以下の句構造を欠損させた欠損質問Ｑを作成すれば良い。 Also, if the correct question Q true is in a language such as English, the missing question Q is created by performing phrase structure analysis, dependency tree analysis, etc., and performing loss in sections or words from this analysis result. Just do it. For example, when the correct question Q true is English, a missing question Q in which a phrase structure below a noun phrase (NP) is missing may be created.
　なお、欠損質問作成部３００は、正解質問Ｑｔｒｕｅの構文情報が破壊された欠損質問Ｑは作成しないことが好ましい。例えば、正解質問Ｑｔｒｕｅが「プランＡの料金を教えて」であり、係り受け解析の解析結果を用いる場合、係り受け関係にない「プランＡを教えて」との欠損質問Ｑは作成しないことが好ましい。 Note that it is preferable that the missing question creation unit 300 does not create the missing question Q in which the syntax information of the correct answer question Q true is destroyed. For example, if the correct answer Q true is “Tell me the price for plan A” and use the analysis result of dependency analysis, do not create the missing question Q “Tell me about plan A”, which is not related to dependency. Is preferred.
　また、欠損質問作成部３００は、例えば、パターンマッチングにより欠損質問Ｑを作成しても良い。例えば、所定の表現をマーカに用いて、正解質問Ｑｔｒｕｅにおける欠損位置を決定する等である。具体的には、例えば、所定の表現として「～の場合」をマーカに用いることが考えられる。この場合、正解質問Ｑｔｒｕｅが「契約が２年未満の場合の違約金は？」であったとすれば、マーカ「～の場合」よりも前の文を欠損させた欠損質問Ｑ「違約金は？」を作成することができる。 The missing question creation unit 300 may create the missing question Q by pattern matching, for example. For example, the missing position in the correct answer question Q true is determined using a predetermined expression as a marker. Specifically, for example, it is conceivable to use “in the case of” as a marker as a predetermined expression. In this case, if the correct answer Q true is “What is the penalty for contracts of less than 2 years?”, The missing question Q “The penalty for the penalty” Can be created.
　ステップＳ２０３：改訂質問生成モデル学習部４００の照合部２１０は、マッチング情報を生成する。このステップＳ２０３は、図４のステップＳ１０２における入力質問Ｑを欠損質問Ｑと読み替えることで、ステップＳ１０２と同様であるため、その説明を省略する。 Step S203: The verification unit 210 of the revised question generation model learning unit 400 generates matching information. Since this step S203 is the same as step S102 by replacing the input question Q in step S102 of FIG. 4 with the missing question Q, description thereof is omitted.
　ステップＳ２０４：改訂質問生成モデル学習部４００の質問復元部２２０は、改訂質問ＲＱを生成する。このステップＳ２０４は、図４のステップＳ１０３における入力質問Ｑを欠損質問Ｑと読み替えることで、ステップＳ１０３と同様であるため、その説明を省略する。 Step S204: The question restoration unit 220 of the revised question generation model learning unit 400 generates a revised question RQ. Since this step S204 is the same as step S103 by replacing the input question Q in step S103 of FIG. 4 with the missing question Q, description thereof is omitted.
　ステップＳ２０５：改訂質問生成モデル学習部４００のパラメータ更新部４１０は、ミニバッチに含まれる各学習データを用いてそれぞれ生成された改訂質問ＲＱと、当該学習データに含まれる正解質問Ｑｔｒｕｅとの誤差を計算する。誤差の計算に用いられる誤差関数としては、例えば、クロスエントロピーを用いれば良い。なお、誤差関数は、改訂質問生成モデルに応じて適宜に決定される。 Step S205: The parameter update unit 410 of the revised question generation model learning unit 400 calculates an error between the revised question RQ generated using each learning data included in the mini-batch and the correct question Q true included in the learning data. calculate. For example, cross-entropy may be used as an error function used for error calculation. The error function is appropriately determined according to the revised question generation model.
　ステップＳ２０６：改訂質問生成モデル学習部４００のパラメータ更新部４１０は、上記のステップＳ２０５で計算した誤差を用いて、改訂質問生成モデルのパラメータを更新する。すなわち、パラメータ更新部４１０は、例えば、上記のステップＳ２０５で計算した誤差を用いて、誤差逆伝播法（バックプロパゲーション）により誤差関数の偏微分値を計算することで、改訂質問生成モデルのパラメータを更新する。これにより、改訂質問生成モデルが学習される。 Step S206: The parameter update unit 410 of the revised question generation model learning unit 400 updates the parameters of the revised question generation model using the error calculated in step S205. That is, for example, the parameter update unit 410 calculates the partial differential value of the error function by the error back-propagation method (back propagation) using the error calculated in step S205 described above. Update. Thereby, the revised question generation model is learned.
　ここで、図５に示す改訂質問生成モデルのパラメータを更新する場合に用いる誤差関数について説明する。 Here, the error function used when updating the parameters of the revised question generation model shown in FIG. 5 will be described.
　図５に示す改訂質問生成モデルでは、生成確率Ｐで生成される各単語ｙｓが、正解質問Ｑｔｒｕｅと一致するようにパラメータ（以降、学習対象のパラメータを「θ」と表す。）を学習する必要がある。ここで、単語ｙｓの生成確率Ｐは、上記の式（１８）に示す通り、適切なλｓが設定されている必要がある。そこで、本発明の第一の実施形態では、単語ｙｓの生成確率Ｐとλｓとを同時に学習するマルチタスク学習により改訂質問生成モデルを学習するものとし、誤差関数は、単語ｙｓの生成確率Ｐに関する誤差Ｌｇと、λｓに関する誤差Ｌλとの和Ｌ（θ）＝Ｌｇ＋Ｌλとする。この誤差関数Ｌを最小化するように、パラメータθを更新する。 In the revised question generation model shown in FIG. 5, the parameters (hereinafter, the learning target parameter is expressed as “θ”) so that each word y s generated with the generation probability P matches the correct question Q true . There is a need to. Here, the generation probability P of the word y s needs to be set to an appropriate λ s as shown in the above equation (18). Therefore, in the first embodiment of the present invention, shall learn the revision question generator model multitask learning for learning the generation probability P and lambda s of words y s simultaneously, the error function is the generation of a word y s The sum L (θ) = L g + L λ of the error L g related to the probability P and the error L λ related to λ s is assumed. The parameter θ is updated so that the error function L is minimized.
　ここで、λｓは、１に近い値を取る程、関連文書Ｘに含まれる単語がｙｓとしてコピーされる確率が高くなることを示している。前記したように、学習時には、正解データとして用いる入力質問Ｑを構成する各単語に対して、該単語が関連文書Ｘに含まれる単語である場合は１、それ以外の場合は０となるラベルを付与するものとする。このラベルを正解としてλｓを生成するニューラルネットワークの学習を行うことで、λｓは、ｃ＾ｓが生成する単語ｙｓが関連文書Ｘに含まれる単語であるか否かを予測する確率となる。この学習によって、改訂質問ＲＱの生成時には、λｓが１に近い値である程、生成されて欲しい単語が関連文書Ｘ内にある確率が高いと判断され、生成確率Ｐｃが強く考慮されるようになる。 Here, lambda s is enough to take a value close to 1 indicates that the probability of the words contained in related document X is copied as y s is increased. As described above, at the time of learning, for each word constituting the input question Q used as correct answer data, a label that is 1 if the word is a word included in the related document X, and a label that is 0 otherwise. Shall be granted. By learning a neural network that generates λ s with this label as a correct answer, λ s is a probability of predicting whether the word y s generated by c ^ s is a word included in the related document X, and Become. With this learning, when the revised question RQ is generated, it is determined that the closer λ s is to 1, the higher the probability that the word desired to be generated is in the related document X, and the generation probability P c is strongly considered. It becomes like this.
　上記の誤差関数Ｌ（θ）＝Ｌｇ＋Ｌλにおける誤差Ｌλ、Ｌｇは、ニューラルネットワークの学習における一般的な方法で計算すれば良い。例えば、誤差Ｌλは２値のクロスエントロピー、誤差Ｌｇは負の対数尤度等を用いて計算することができる。 The errors L λ and L g in the above error function L (θ) = L g + L λ may be calculated by a general method in learning of a neural network. For example, the error L λ can be calculated using binary cross-entropy, and the error L g can be calculated using negative log likelihood.
　（改訂質問生成モデルの変形例）
　ここで、本発明の第一の実施形態では、図５に示す改訂質問生成モデルにより改訂質問生成部２００が実現される場合について説明したが、例えば、図７に示す改訂質問生成モデルや図８に示す改訂質問生成モデルにより改訂質問生成部２００が実現されても良い。 (Modified example of revised question generation model)
Here, in the first embodiment of the present invention, the case where the revised question generation unit 200 is realized by the revised question generation model shown in FIG. 5 has been described. For example, the revised question generation model shown in FIG. The revised question generation unit 200 may be realized by the revised question generation model shown in FIG.
　図７に示す改訂質問生成モデルは、Decode Layerにおいて、生成確率ＰＣ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）を計算する機構を有しないモデルである。この場合、単語ｙｓの最終的な生成確率Ｐ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）＝ＰＧ（ｙｓ｜ｙ＜ｓ，Ｘ，Ｑ）となる。 The revised question generation model shown in FIG. 7 is a model that does not have a mechanism for calculating the generation probability P C (y s | y <s , X, Q) in the decode layer. In this case, the word y s final generation probability P (y s | y <s , X, Q) = P G (y s | y <s, X, Q) become.
　図８に示す改訂質問生成モデルは、図７に示す改訂質問生成モデルに対して、更に、Matching Layerを有しないモデルである。この場合、Decode Layerの処理として、注意機構（attention）では、マッチング行列Ｍの代わりにコンテキスト行列Ｈを用いて、Decoderへの入力ｚ＾ｓが計算される。 The revised question generation model shown in FIG. 8 is a model that does not have a matching layer in addition to the revised question generation model shown in FIG. In this case, as processing of the Decode Layer, the attention mechanism (attention) calculates the input z ^ s to the Decoder using the context matrix H instead of the matching matrix M.
　（質問生成装置１００の機能構成の変形例）
　ここで、改訂質問ＲＱの生成時に、入力質問Ｑと関連する関連文書Ｘが明確ではなく、関連文書Ｘが含まれていると想定される文書集合しか得られない場合がある。このような場合に、文書集合に含まれる各文書を用いて、改訂質問の生成処理を行うとすれば、処理時間が増大する。そこで、改訂質問の処理の前処理として、文書集合から関連文書Ｘを検索する処理を行うことが考えられる。 (Modification of the functional configuration of the question generation device 100)
Here, when the revised question RQ is generated, the related document X related to the input question Q is not clear, and only a document set that is assumed to include the related document X may be obtained. In such a case, if the revision question generation process is performed using each document included in the document set, the processing time increases. In view of this, it is conceivable to perform a process of retrieving the related document X from the document set as a pre-process of the revision question process.
　上記の前処理を行う質問生成装置１００の機能構成を図９に示す。図９は、本発明の第一の実施形態における改訂質問生成時の質問生成装置１００の機能構成の変形例を示す図である。 FIG. 9 shows a functional configuration of the question generation device 100 that performs the above preprocessing. FIG. 9 is a diagram illustrating a modification of the functional configuration of the question generation device 100 when generating a revised question in the first embodiment of the present invention.
　図９に示すように、改訂質問生成時の質問生成装置１００は、更に、関連文書検索部６００を有していても良い。関連文書検索部６００は、入力質問Ｑと、文書集合Ｙと入力して、文書集合Ｙの中から、当該入力質問Ｑに関連する文書（関連文書）Ｘを検索する。そして、関連文書検索部６００は、検索した関連文書Ｘを改訂質問生成部２００に出力する。これにより、関連文書Ｘが含まれていると想定される文書集合しか得られない場合であっても、容易に改訂質問ＲＱを得ることができるようになる。 As shown in FIG. 9, the question generation device 100 when generating a revised question may further include a related document search unit 600. The related document search unit 600 inputs the input question Q and the document set Y, and searches the document set Y for a document (related document) X related to the input question Q. Then, the related document search unit 600 outputs the searched related document X to the revised question generation unit 200. Thereby, even when only a document set that is assumed to include the related document X can be obtained, the revised question RQ can be easily obtained.
　なお、関連文書検索部６００による検索手法は、任意の検索手法を用いることができる。例えば、文書集合Ｙに含まれる各文書と、入力質問Ｑとのスコアをそれぞれ算出した上で、スコアが上位のＮ´件を関連文書Ｘとすることが挙げられる。Ｎ´の値は任意に設定されるが、例えば、１～１０程度等が考えられる。 Note that any search method can be used as the search method by the related document search unit 600. For example, after calculating the scores of each document included in the document set Y and the input question Q, N ′ cases with the highest scores are used as the related documents X. Although the value of N ′ is arbitrarily set, for example, about 1 to 10 can be considered.
　ここで、関連文書検索部６００により検索された関連文書Ｘと、この関連文書Ｘ及び入力質問Ｑから生成された改訂質問ＲＱとを、入力質問Ｑを行った質問者（ユーザ）に提示することも考えられる。そこで、図９に示すように、改訂質問生成時の質問生成装置１００は、更に、表示制御部７００を有していても良い。表示制御部７００は、関連文書検索部６００により検索された関連文書Ｘと、この関連文書Ｘ及び入力質問Ｑから改訂質問生成部２００によって生成された改訂質問ＲＱとを表示する。 Here, the related document X searched by the related document search unit 600 and the revised question RQ generated from the related document X and the input question Q are presented to the questioner (user) who has made the input question Q. Is also possible. Therefore, as illustrated in FIG. 9, the question generation device 100 when generating a revised question may further include a display control unit 700. The display control unit 700 displays the related document X searched by the related document search unit 600 and the revised question RQ generated by the revised question generating unit 200 from the related document X and the input question Q.
　（応用例）
　ここで、上述したように、例えばＮ´として２以上の値が設定された場合等に、文書集合Ｙから複数の関連文書Ｘが得られることがある。この場合、これらの複数の関連文書Ｘのそれぞれを用いて、改訂質問ＲＱを生成することができる。 (Application examples)
Here, as described above, for example, when two or more values are set as N ′, a plurality of related documents X may be obtained from the document set Y. In this case, the revised question RQ can be generated using each of the plurality of related documents X.
　例えば、文書集合Ｙから２つの関連文書Ｘ１及び関連文書Ｘ２が得られた場合、改訂質問生成部２００により、入力質問Ｑ及び関連文書Ｘ１を用いた改訂質問ＲＱ１と、入力質問Ｑ及び関連文書Ｘ２を用いた改訂質問ＲＱ２とが得られる。 For example, if two related documents X 1 and related documents X 2 are obtained from the document set Y, the revised question generator 200, the revised question RQ 1 using the input question Q and related documents X 1, input question Q And a revised question RQ 2 using the related document X 2 is obtained.
　そこで、このような質問生成装置１００の応用例として、ユーザから何等かの質問（入力質問Ｑ）がなされた場合に、複数の改訂質問ＲＱと、この改訂質問ＲＱの生成に用いられた関連文書Ｘとをユーザに提示するチャットボットが考えられる。 Therefore, as an application example of such a question generation device 100, when any question (input question Q) is made by the user, a plurality of revised questions RQ and related documents used for generating this revised question RQ A chat bot that presents X to the user can be considered.
　例えば、図１０に示すように、ユーザから入力質問Ｑ「料金が知りたい」が入力された場合（Ｓ１１）、質問生成装置１００の関連文書検索部６００は、文書集合Ｙから複数の関連文書Ｘ（関連文書Ｘ１及び関連文書Ｘ２）を検索する。そして、質問生成装置１００の表示制御部７００は、関連文書Ｘ１及び入力質問Ｑから改訂質問生成部２００によって生成された改訂質問ＲＱ１「プランＡの料金が知りたい」と、関連文書Ｘ１へのリンクと、関連文書Ｘ２及び入力質問Ｑから改訂質問生成部２００によって生成された改訂質問ＲＱ２「特別割引が適用されたときの料金が知りたい」と、関連文書Ｘ２へのリンクとをユーザに表示する（Ｓ１２）。これにより、ユーザが曖昧な質問（入力質問Ｑ）を行った場合であっても、質問生成装置１００は、複数の改訂質問ＲＱと、これらの複数の改訂質問ＲＱにそれぞれ関連する関連文書Ｘへのリンクとをユーザに提示することができるようになる。 For example, as shown in FIG. 10, when an input question Q “I want to know the fee” is input from the user (S 11), the related document search unit 600 of the question generation device 100 reads a plurality of related documents X from the document set Y. Search for (related documents X 1 and related documents X 2). Then, the display control unit 700 of the question generator 100, and related documents X 1 and input question generated by the revised question generator 200 from Q revised question RQ 1 "want to know Price Plan A", related document X 1 and links to, and related documents X 2 and input question Q generated by the revised question generation unit 200 from the revised question RQ 2 "I want to know the price at the time when the special discount is applied", links to related documents X 2 Are displayed to the user (S12). As a result, even when the user makes an ambiguous question (input question Q), the question generating device 100 moves to a plurality of revised questions RQ and related documents X respectively related to the plurality of revised questions RQ. Can be presented to the user.
　また、チャットボットへの他の応用例として、複数の改訂質問ＲＱ及び関連文書Ｘを順に提示しても良い。例えば、図１１に示すように、ユーザから入力質問Ｑ「料金が知りたい」が入力された場合（Ｓ２１）、質問生成装置１００の関連文書検索部６００は、文書集合Ｙから複数の関連文書Ｘ（関連文書Ｘ１及び関連文書Ｘ２）を検索する。そして、質問生成装置１００の表示制御部７００は、例えば、改訂質問ＲＱ１「プランＡの料金が知りたい」を意図しているか否かをユーザに対して確認するための文を表示する（Ｓ２２）。 Further, as another application example to the chatbot, a plurality of revised questions RQ and related documents X may be presented in order. For example, as illustrated in FIG. 11, when the input question Q “I want to know the fee” is input from the user (S 21), the related document search unit 600 of the question generation device 100 reads a plurality of related documents X from the document set Y. Search for (related documents X 1 and related documents X 2). Then, the display control unit 700 of the question generation device 100 displays, for example, a sentence for confirming to the user whether or not the revised question RQ 1 “I want to know the charges for Plan A” is intended (S22). ).
　この確認文に対して、ユーザから「違うよ」等の否定を示す応答が入力された場合（Ｓ２３）、質問生成装置１００の表示制御部７００は、例えば、改訂質問ＲＱ２「特別割引が適用されたときの料金が知りたい」を意図しているか否かをユーザに対して確認するための文を表示する（Ｓ２４）。 When a negative response such as “No” is input from the user in response to the confirmation text (S23), the display control unit 700 of the question generation device 100 may display, for example, the revised question RQ 2 “Special discount applied” A sentence for confirming to the user whether or not the user wants to know the charge when the service is made is displayed (S24).
　この確認文に対して、ユーザから「そうだよ」等の肯定を示す応答が入力された場合（Ｓ２５）、質問生成装置１００の表示制御部７００は、例えば、関連文書Ｘ２へのリンクをユーザに提示する（Ｓ２６）。 For this confirmation text, if the response indicating affirmation such as "That's right" is input from the user (S25), the display control unit 700 of the question generator 100, for example, user links to related documents X 2 (S26).
　これにより、ユーザが曖昧な質問（入力質問Ｑ）を行った場合であっても、質問生成装置１００は、対話的に、改訂質問ＲＱと、この改訂質問ＲＱに関連する関連文書Ｘへのリンクとをユーザに提示することができるようになる。 Thus, even when the user makes an ambiguous question (input question Q), the question generation device 100 interactively links the revised question RQ and the related document X related to the revised question RQ. Can be presented to the user.
　（まとめ）
　以上のように、本発明の第一の実施形態における質問生成装置１００は、例えばニューラルネットワークにより実現される改訂質問生成モデルを用いて、潜在的な欠損が含まれている可能性がある入力質問Ｑから、欠損が含まれない改訂質問ＲＱを生成することができる。これにより、例えば、改訂質問ＲＱを用いた質問応答タスク等を行う場合に、当該質問応答タスクの回答精度を向上させることができるようになる。 (Summary)
As described above, the question generation device 100 according to the first embodiment of the present invention uses, for example, a revised question generation model realized by a neural network, and an input question that may contain a potential defect. From Q, a revised question RQ that does not include a deficit can be generated. Thereby, for example, when a question answering task using the revised question RQ is performed, the answer accuracy of the question answering task can be improved.
　また、本発明の第一の実施形態における質問生成装置１００では、改訂質問生成モデルを用いて改訂質問ＲＱを生成する際に、入力質問Ｑに関連する関連文書Ｘに含まれる単語をコピーした改訂質問ＲＱを生成する。これにより、上記の質問応答タスクの回答精度を更に向上させることができると共に、ユーザは、関連文書Ｘのどの部分から改訂質問ＲＱが生成されたのかを知ることができるようになる。 Further, in the question generation device 100 according to the first embodiment of the present invention, when the revised question RQ is generated using the revised question generation model, the revision in which a word included in the related document X related to the input question Q is copied. A question RQ is generated. As a result, the answer accuracy of the question answering task can be further improved, and the user can know from which part of the related document X the revised question RQ is generated.
　また、本発明の第一の実施形態における質問生成装置１００では、１つの入力質問Ｑに対して複数のバリエーションの改訂質問ＲＱを生成することもできる。例えば、本発明の第一の実施形態における質問生成装置１００では、１つの入力質問Ｑ「料金を知りたい」に対して、改訂質問Ｑとして、「プランＡの料金を知りたい」、「特別割引が適用されたときの料金を知りたい」等のバリエーションを生成することができる。これにより、例えば、複数のバリエーションの改訂質問Ｑの中から質問の意図に近い改訂質問Ｑをユーザに選択させることもできるようになる。 Also, the question generation device 100 according to the first embodiment of the present invention can generate a plurality of variations of revised questions RQ for one input question Q. For example, in the question generation device 100 according to the first embodiment of the present invention, for one input question Q “I want to know the fee”, as the revised question Q, “I want to know the fee for Plan A”, “Special discount” Variations such as “I want to know the fee when is applied” can be generated. Thereby, for example, it becomes possible to allow the user to select a revised question Q close to the intention of the question from among a plurality of revised questions Q.
　更に、１つの入力質問Ｑに対して複数のバリエーションの改訂質問ＲＱを生成することで、本発明の第一の実施形態における質問生成装置１００は、例えば、「よくある質問集（ＦＡＱ）」を自動で作成したり、拡張したりすること等にも応用することができる。 Further, by generating a plurality of variations of revised questions RQ for one input question Q, the question generation device 100 according to the first embodiment of the present invention can, for example, generate a “common question collection (FAQ)”. It can also be applied to automatic creation and expansion.
　［第二の実施形態］
　次に、本発明の第二の実施形態について説明する。 [Second Embodiment]
Next, a second embodiment of the present invention will be described.
　（概要）
　上述した第一の実施形態では、入力質問と関連文書とが与えられた場合に、質問生成装置１００が改訂質問生成モデルを用いて、当該入力質問の改訂質問を生成する場合について説明した。しかしながら、例えば、入力質問が短い場合や曖昧なものである場合等には、当該入力質問に対する回答が一意に特定できるとは限らず、回答の可能性が関連文書中に複数箇所存在することがある。したがって、このような場合に、回答を考慮せずに質問を詳細化・具体化した場合、回答不能な改訂質問が生成されることがある。また、複数パターンの詳細化・具体化をした場合であっても、全ての改訂質問に対する回答が同じになってしまう可能性も想定される。更に、機械読解等の質問応答技術で回答できるのは１つのみ（つまり、１問１答）であることが多く、回答が複数想定されるような質問には完全には対応することができない。 (Overview)
In the first embodiment described above, the case where the question generation device 100 generates the revised question of the input question using the revised question generation model when the input question and the related document are given has been described. However, for example, when the input question is short or ambiguous, the answer to the input question may not be uniquely identified, and there may be multiple possible answers in the related document. is there. Therefore, in such a case, if the question is refined and embodied without considering the answer, a revised question that cannot be answered may be generated. Moreover, even when a plurality of patterns are refined and embodied, there is a possibility that answers to all revised questions will be the same. Furthermore, question answering techniques such as machine reading can often answer only one question (that is, one answer per question), and it is not possible to completely respond to a question that has multiple possible answers. .
　そこで、本発明の第二の実施形態では、入力質問と関連文書とが与えられた場合に、質問生成装置１００が改訂質問を生成する前に、質問応答を行って、入力質問に対してＮ個（Ｎは１以上の整数）の回答を生成する。そして、質問生成装置１００は、これらＮ個の回答の各々に対して、改訂質問を生成する。これにより、入力質問に対する回答が複数存在する場合であっても、これらの回答を機械読解等でそれぞれ一意に得るための改訂質問を生成することができ、短い質問や曖昧な質問に対しても高い回答精度を実現することができる。なお、質問応答によって生成されるＮ個の回答は、入力質問に対する最終的な回答（つまり、質問者が真に必要とする回答）の候補となるため、「回答候補」とも表す。 Therefore, in the second embodiment of the present invention, when an input question and a related document are given, a question response is made before the question generation device 100 generates a revised question, and N is input to the input question. Number of answers (N is an integer of 1 or more) is generated. Then, the question generation device 100 generates a revised question for each of these N answers. As a result, even if there are multiple answers to the input question, it is possible to generate a revised question for uniquely obtaining these answers by machine reading etc., even for short questions and ambiguous questions. High response accuracy can be achieved. Note that the N answers generated by the question answer are candidates for the final answer to the input question (that is, the answer that the questioner really needs), and are also referred to as “answer candidates”.
　本発明の第二の実施形態における改訂質問の生成について、図１２を参照しながら、より具体的に説明する。例えば、図１２に示す関連文書と、入力質問「午後５時時点の円相場はどうなりましたか？」とが与えられたとする。この場合、関連文書中には、入力質問に対する回答候補が複数存在する（つまり、関連文書には、当該入力質問に対する回答候補として、ドルに対する円相場の情報と、ユーロに対する円相場の情報とが記載されている。）。したがって、この時点では、これら複数の回答候補のうちのどの回答候補が、質問者が真に必要とする回答であるのかを判定することはできない。 The generation of the revision question in the second embodiment of the present invention will be described more specifically with reference to FIG. For example, it is assumed that the related document shown in FIG. 12 and the input question “What happened to the yen exchange rate at 5:00 pm?” Are given. In this case, there are a plurality of answer candidates for the input question in the related document (that is, in the related document, as the answer candidate for the input question, the yen exchange rate information for the dollar and the yen exchange rate information for the euro are included. Has been described.). Therefore, at this time, it cannot be determined which answer candidate among the plurality of answer candidates is the answer that the questioner really needs.
　そこで、本発明の第二の実施形態では、まず、回答１「先週末と比べて２６銭円高ドル安の１ドル＝１０９円７４銭から７５銭」及び回答２「先週末と比べて６４銭円安ユーロ高の１ユーロ＝１２９円５７銭から６１銭」の２つの回答候補を生成する。そして、これらの回答を用いて、当該回答を一意に決定できるような質問となるように入力質問を詳細化・具体化することで、それぞれの回答に対して改訂質問を生成する。図１２に示す例では、入力質問に対して「ドルに対して」と「ユーロに対して」とをそれぞれ付与して、改訂質問１「午後５時時点の円相場はドルに対してどうなりましたか？」及び改訂質問２「午後５時時点の円相場はユーロに対してどうなりました？」を生成する。 Therefore, in the second embodiment of the present invention, firstly, the answer 1 “1 dollar = 26.75 yen to 75 yen from 26 dollars higher than last weekend” and answer 2 “64 compared to last weekend”. Two answer candidates are generated: 1 Euro = 129.57 yen to 61 yen with a weak yen against the euro. Then, using these answers, the input question is refined and specified so that the answer can be uniquely determined, and a revised question is generated for each answer. In the example shown in FIG. 12, “To dollar” and “To euro” are assigned to the input question, respectively, and revised question 1 “What happens to the yen at 5 pm against the dollar? And revised question 2 “What happened to the euro at 5pm against the euro?”.
　このように、本発明の第二の実施形態では、以下の（１）及び（２）により改訂質問を生成する。 Thus, in the second embodiment of the present invention, the revised question is generated by the following (1) and (2).
　（１）入力質問に対して質問応答を行って、当該入力質問に対する回答（回答候補）をＮ個生成する。 (1) A question response is made to the input question, and N answers (answer candidates) for the input question are generated.
　（２）Ｎ個の回答毎に、当該回答を得るための改訂質問を生成する（すなわち、Ｎ個の回答それぞれに対応するＮ個の改訂質問を生成する。）。 (2) For each N answers, a revised question for generating the answer is generated (that is, N revised questions corresponding to each of the N answers are generated).
　ここで、上記の（１）及び（２）は、ニューラルネットワークで実現された改訂質問生成モデルにより、ｅｎｄ－ｔｏ－ｅｎｄで同時に実行することができる。ただし、改訂質問生成モデルは、必ずしもニューラルネットワークで実現される必要はなく、改訂質問生成モデルの全部又は一部がニューラルネットワーク以外の機械学習モデルで実現されていても良い。また、上記の（１）の質問応答を行うモデルと、上記の（２）の改訂質問を生成するモデルとを別々に用意して、これらを個別に又は組み合わせて用いても良い。 Here, the above (1) and (2) can be executed simultaneously end-to-end by the revised question generation model realized by the neural network. However, the revised question generation model is not necessarily realized by a neural network, and all or a part of the revised question generation model may be realized by a machine learning model other than the neural network. Also, the model that performs the question response of (1) above and the model that generates the revised question of (2) above may be prepared separately and used individually or in combination.
　上記の（１）の質問応答では、関連文書から回答（回答候補）となる可能性が高い情報を発見し、この発見した情報をベースに回答を行う。ここで、回答（回答候補）を得る方法としては、例えば、関連文書中の記述をそのまま抽出したものを回答とする方法や関連文書中の記述を参考に回答となる文を生成する方法等、種々の方法が存在する。本発明の第二の実施形態では、一例として、上記の（１）で回答（回答候補）を得る方法として、主に、関連文書中の記述をそのまま抽出したものを回答とする方法を用いる場合について説明する。 In the above question response (1), information that is highly likely to be an answer (answer candidate) is found from related documents, and an answer is made based on the found information. Here, as a method of obtaining an answer (answer candidate), for example, a method in which a description in a related document is extracted as it is, a method of generating a sentence that becomes an answer with reference to the description in the related document, etc. There are various methods. In the second embodiment of the present invention, as an example, as a method for obtaining an answer (answer candidate) in the above (1), a method in which an answer obtained by extracting a description in a related document as it is is used. Will be described.
　ここで、改訂質問生成モデルの学習では、第一の実施形態と同様に、正解データとして用いる入力質問と、この入力質問の一部を欠損させた質問（つまり、欠損質問）と、関連文書とを入力として、欠損質問と関連文書とを用いて得られる自然文が、正解データである入力質問に近付くように改訂質問生成モデルのパラメータを更新する。このとき、改訂質問生成モデルの内部では、第一の実施形態と同様に、欠損質問と関連文書とのマッチングが行われ、欠損部分を関連文書から発見して補われる。このような改訂質問生成モデルが学習されることで、第一の実施形態と同様に、例えば、自然文の短い入力質問と関連文書とが入力された場合に、当該入力質問の潜在的に欠損した部分が関連文書から発見及び補われ、入力質問よりもより詳細化及び具体化した改訂質問文が生成される。 Here, in the learning of the revised question generation model, as in the first embodiment, an input question used as correct answer data, a question in which a part of the input question is missing (that is, a missing question), a related document, And the parameters of the revised question generation model are updated so that the natural sentence obtained using the missing question and the related document approaches the input question which is correct answer data. At this time, in the revised question generation model, similar to the first embodiment, matching between the missing question and the related document is performed, and the missing portion is found from the related document and compensated. By learning such a revised question generation model, as in the first embodiment, for example, when an input question with a short natural sentence and a related document are input, the input question is potentially missing. The revised part is found and supplemented from the related document, and a revised question sentence that is more detailed and embodied than the input question is generated.
　また、第二の実施の形態では、改訂質問生成モデルの学習において、入力質問に対する回答の正解を正解データとして、当該入力質問に対する回答が正解データに近付くように改訂質問生成モデルのパラメータを更新する。 In the second embodiment, in the revised question generation model learning, the correct answer to the input question is used as the correct answer data, and the parameters of the revised question generation model are updated so that the answer to the input question approaches the correct answer data. .
　（質問生成装置１００の機能構成）
　まず、本発明の第二の実施形態における改訂質問生成時の質問生成装置の機能構成について、図１３を参照しながら説明する。図１３は、本発明の第二の実施形態における改訂質問生成時の質問生成装置１００の機能構成の一例を示す図である。 (Functional configuration of the question generation device 100)
First, the functional configuration of the question generation device when generating a revised question in the second embodiment of the present invention will be described with reference to FIG. FIG. 13 is a diagram illustrating an example of a functional configuration of the question generation device 100 when generating a revised question in the second embodiment of the present invention.
　図１３に示すように、本発明の第二の実施形態における質問生成装置１００は、テキスト処理部８００と、改訂質問生成部９００と、出力部１０００とを有する。 As illustrated in FIG. 13, the question generation device 100 according to the second embodiment of the present invention includes a text processing unit 800, a revised question generation unit 900, and an output unit 1000.
　テキスト処理部８００は、自然文で記述された入力質問と関連文書とを入力して、これらの入力質問及び関連文書を改訂質問生成部９００に入力するための前処理を行う。具体的には、テキスト処理部８００は、例えば形態素解析等を行うことによって、自然文で記述された入力質問及び関連文書をそれぞれ単語トークンの集合（単語系列）に変換する。なお、入力質問及び関連文書の少なくとも一方が音声認識結果として得られた文等であっても良い。また、テキスト処理部８００に入力される関連文書は、１つ以上の文書（すなわち、関連文書の集合）であっても良い。本発明の第二の実施形態では、「関連文書」と表した場合には、関連文書の集合も含まれるものとする。 The text processing unit 800 inputs an input question and a related document described in a natural sentence, and performs preprocessing for inputting the input question and the related document to the revised question generating unit 900. Specifically, the text processing unit 800 converts an input question and a related document described in a natural sentence into a set of word tokens (word series) by performing, for example, morphological analysis. Note that at least one of the input question and the related document may be a sentence obtained as a speech recognition result. The related document input to the text processing unit 800 may be one or more documents (that is, a set of related documents). In the second embodiment of the present invention, when “related documents” is represented, a set of related documents is also included.
　また、以降では、第一の実施形態と同様に、入力質問はＪ個の単語トークンの集合（単語系列）Ｑ＝｛ｑ０，ｑ１，・・・，ｑＪ｝に変換されるものとし、この単語系列Ｑも入力質問Ｑと表すものとする。同様に、関連文書はＴ個の単語トークンの集合（単語系列）Ｘ＝｛ｘ０，ｘ１，・・・，ｘＴ｝に変換されるものとし、この単語系列Ｘも関連文書Ｘと表すものとする。 In the following, as in the first embodiment, the input question is converted into a set of J word tokens (word sequence) Q = {q 0 , q 1 ,..., Q J }. This word sequence Q is also expressed as an input question Q. Similarly, it is assumed that the related document is converted into a set of T word tokens (word sequence) X = {x 0 , x 1 ,..., X T }. Shall.
　なお、単語系列で表された入力質問Ｑ及び関連文書Ｘが質問生成装置１００に入力される場合は、当該質問生成装置１００はテキスト処理部８００を有しなくても良い。 Note that when the input question Q and the related document X expressed in word series are input to the question generation device 100, the question generation device 100 may not include the text processing unit 800.
　改訂質問生成部９００は、入力質問に対する質問応答と、当該質問応答によって得られた回答（回答候補）に対応する改訂質問の生成とを行う。改訂質問生成部９００は、学習済みの改訂質問生成モデル（すなわち、後述する改訂質問生成モデル学習部１１００によって更新されたパラメータを用いた改訂質問生成モデル）により実現される。 The revised question generation unit 900 generates a question response to the input question and a revised question corresponding to the answer (answer candidate) obtained by the question response. The revised question generation unit 900 is realized by a learned revised question generation model (that is, a revised question generation model using parameters updated by a revised question generation model learning unit 1100 described later).
　ここで、改訂質問生成部９００には、質問応答実行部９１０と、質問生成部９２０とが含まれる。 Here, the revised question generation unit 900 includes a question response execution unit 910 and a question generation unit 920.
　質問応答実行部９１０は、入力質問Ｑと、関連文書Ｘとを入力して、質問応答を行って、当該入力質問Ｑに対する回答候補を関連文書Ｘから生成する。なお、上述したように、ここで生成される回答候補は１つである必要はなく、Ｎを１以上の整数としてＮ個の回答候補が生成される。本発明の第二の実施形態では、関連文書中の記述をそのまま抽出したものを回答候補とする方法を用いるが、これに限られず、自然文の質問と任意の文書（関連文書）とを入力として自然文の回答を得ることができる方法であれば任意の方法を用いることができる。 The question response execution unit 910 inputs the input question Q and the related document X, performs a question response, and generates answer candidates for the input question Q from the related document X. As described above, the number of answer candidates generated here is not necessarily one, and N answer candidates are generated with N being an integer of 1 or more. In the second embodiment of the present invention, a method in which the description in the related document is extracted as it is is used as the answer candidate. However, the present invention is not limited thereto, and a natural sentence question and an arbitrary document (related document) are input Any method can be used as long as a natural sentence answer can be obtained.
　質問生成部９２０は、入力質問Ｑと、関連文書Ｘと、Ｎ個の回答候補とを入力して、当該入力質問Ｑよりも詳細化・具体化した改訂質問ＲＱを生成する。このとき、質問生成部９２０は、Ｎ個の回答候補の各々に対して改訂質問ＲＱを生成する（すなわち、Ｎ個の回答候補のそれぞれに対応するＮ個の改訂質問ＲＱを生成する。）。 The question generation unit 920 inputs the input question Q, the related document X, and N answer candidates, and generates a revised question RQ that is more detailed and specific than the input question Q. At this time, the question generation unit 920 generates a revised question RQ for each of the N answer candidates (that is, generates N revised questions RQ corresponding to each of the N answer candidates).
　ここで、本発明の第二の実施形態では、質問生成部９２０は、各回答候補をそれぞれ一意に特定可能とするような情報を入力質問Ｑに対して追加することで改訂質問ＲＱを生成する。例えば、関連文書Ｘ中で回答候補となる情報の周辺には「～の場合」や「～であるときには」といった条件に関する情報が記述されている場合がある。したがって、このような条件に関する情報を入力質問Ｑに追加することで、この条件に合致した場合の回答（回答候補）を一意に決定することができる改訂質問ＲＱを生成することができる。この他にも、例えば、人名や地名等の固有表現も回答候補を絞り込むための有益な情報となり得るので、これらを入力質問Ｑに追加した改訂質問ＲＱを生成しても良い。 Here, in the second embodiment of the present invention, the question generation unit 920 generates the revised question RQ by adding information that can uniquely identify each answer candidate to the input question Q. . For example, information related to conditions such as “in the case of” and “in the case of” may be described in the vicinity of the information that is the answer candidate in the related document X. Therefore, by adding information regarding such conditions to the input question Q, it is possible to generate a revised question RQ that can uniquely determine an answer (answer candidate) when this condition is met. In addition to this, for example, a proper expression such as a person name or a place name can be useful information for narrowing down answer candidates, and a revised question RQ in which these are added to the input question Q may be generated.
　なお、改訂質問ＲＱの生成方法や入力質問Ｑに追加する情報の発見方法、入力質問Ｑへの情報の追加方法等は、上述した「各回答候補をそれぞれ一意に特定可能とするような情報を入力質問Ｑに対して追加することで改訂質問ＲＱを生成する」ものであれば任意の手法を採用することができる。例えば、上述した「～の場合」という情報をパターンマッチングで発見及び抽出した上で、抽出した情報の中から回答（回答候補）に最も近い場所にある情報を、入力質問Ｑの先頭に追加して改訂質問ＲＱを生成する、といった手法を用いても良い。又は、例えば、ニューラルネットワークによる文生成手法を用いて改訂質問ＲＱが生成されても良い。 The generation method of the revised question RQ, the discovery method of information to be added to the input question Q, the method of adding information to the input question Q, and the like are described above. Any method can be employed as long as it can generate a revised question RQ by adding to the input question Q. For example, after finding and extracting the information “in the case of” described above by pattern matching, the information closest to the answer (answer candidate) is added to the head of the input question Q from the extracted information. A method of generating a revised question RQ may be used. Alternatively, for example, the revised question RQ may be generated using a sentence generation method using a neural network.
　出力部１０００は、Ｎ個の回答（回答候補）と、これらのＮ個の回答のそれぞれに対応するＮ個の改訂質問ＲＱとを出力する。このとき、出力部１０００は、例えば、或る回答候補と、この回答候補に対応する改訂質問ＲＱとの組（ペア）を１以上出力する。ここで、回答候補と改訂質問ＲＱとの組（ペア）の出力方法には、質問生成装置１００のユーザインタフェースに応じて任意の方法を採用することができる。 The output unit 1000 outputs N answers (answer candidates) and N revised questions RQ corresponding to each of these N answers. At this time, for example, the output unit 1000 outputs one or more pairs of a certain answer candidate and a revised question RQ corresponding to this answer candidate. Here, as a method for outputting a pair of the answer candidate and the revised question RQ, an arbitrary method can be adopted according to the user interface of the question generation device 100.
　例えば、質問生成装置１００が検索システム等のように画面に回答を出力するユーザインタフェースを備えている場合、ユーザ（質問者）から入力された入力質問Ｑに対して、検索結果のサジェスト機能のように「もしかして・・・」と改訂質問ＲＱの候補を表示し、ユーザによって改訂質問ＲＱが選択されたときに当該改訂質問ＲＱに対応する回答（回答候補）を表示する、といった方法を採用しても良い。 For example, when the question generation device 100 includes a user interface that outputs an answer to the screen as in a search system or the like, a search result suggestion function for an input question Q input from a user (questioner) In such a case, the candidate of the revised question RQ is displayed as "Maybe ...", and when the revised question RQ is selected by the user, an answer (answer candidate) corresponding to the revised question RQ is displayed. Also good.
　また、例えば、質問生成装置１００が音声対話によるユーザインタフェースを備えている場合、ユーザから入力質問Ｑが入力されると、最も尤度の高い回答（回答候補）に対応する改訂質問ＲＱについて「もしかして○○ということですか？」（○○は当該改訂質問ＲＱの質問内容）といったように確認の聞き返しを発話し、ユーザが同意したときに当該改訂質問ＲＱに対応する回答（回答候補）を発話する、といった方法を採用しても良い。なお、このとき、例えば、確認の聞き返しの発話に対してユーザが非同意の場合には次に尤度の高い回答（回答候補）に対応する改訂質問ＲＱについて確認の聞き返しを発話し、ユーザが同意するまでこれを繰り返す、といった方法を採用しても良い。ここで、回答（回答候補）の尤度については、例えば、当該尤度を算出する機能を質問生成装置１００が備えていても良いし、質問応答実行部９１０で回答候補の生成と共に当該回答候補の尤度が算出されても良い。 Further, for example, when the question generating device 100 includes a user interface by voice dialogue, when the input question Q is input from the user, the revised question RQ corresponding to the most likely answer (answer candidate) is “probably. Say “Yes, is it?” (XX is the question content of the revised question RQ), and utters the answer (answer candidate) corresponding to the revised question RQ when the user agrees You may adopt the method of doing. At this time, for example, when the user disagrees with the confirmation utterance, the user confirms the confirmation question about the revised question RQ corresponding to the next most likely answer (answer candidate). You may adopt the method of repeating this until it agrees. Here, regarding the likelihood of the answer (answer candidate), for example, the question generation device 100 may have a function of calculating the likelihood, or the answer candidate is generated together with the generation of the answer candidate by the question answer execution unit 910. Likelihood may be calculated.
　なお、出力部１０００の出力先は上述したものに限られず、例えば、補助記憶装置５０８や記録媒体５０３ａ、ネットワークを介して接続される他の装置等であっても良い。 Note that the output destination of the output unit 1000 is not limited to that described above, and may be, for example, the auxiliary storage device 508, the recording medium 503a, or other devices connected via a network.
　次に、本発明の第二の実施形態における学習時の質問生成装置１００の機能構成について、図１４を参照しながら説明する。図１４は、本発明の第二の実施形態における学習時の質問生成装置１００の機能構成の一例を示す図である。 Next, the functional configuration of the question generation device 100 during learning in the second embodiment of the present invention will be described with reference to FIG. FIG. 14 is a diagram illustrating an example of a functional configuration of the question generation device 100 during learning according to the second embodiment of the present invention.
　図１４に示すように、本発明の第二の実施形態における学習時の質問生成装置１００は、欠損質問作成部３００と、改訂質問生成モデル学習部１１００とを有する。 As shown in FIG. 14, the question generation device 100 at the time of learning in the second embodiment of the present invention includes a missing question creation unit 300 and a revised question generation model learning unit 1100.
　欠損質問作成部３００は、第一の実施形態と同様に、入力質問Ｑを入力して、当該入力質問Ｑの一部を欠損させることで、欠損質問を作成する。 The missing question creating unit 300 creates the missing question by inputting the input question Q and missing a part of the input question Q, as in the first embodiment.
　改訂質問生成モデル学習部１１００は、欠損質問作成部３００が作成した欠損質問と、入力質問Ｑと、この入力質問Ｑに対する正解回答Ａｔｒｕｅと、関連文書Ｘとを用いて、改訂質問生成モデルを学習する。そして、改訂質問生成モデル学習部１１００は、学習済みの改訂質問生成モデルのパラメータを出力する。 The revised question generation model learning unit 1100 generates a revised question generation model using the missing question created by the missing question creation unit 300, the input question Q, the correct answer A true for the input question Q, and the related document X. learn. Then, the revised question generation model learning unit 1100 outputs the learned revised question generation model parameters.
　ここで、改訂質問生成モデル学習部１１００には、質問応答実行部９１０と、質問生成部９２０と、パラメータ更新部１１１０とが含まれる。質問応答実行部９１０及び質問生成部９２０は、上述した通りである。パラメータ更新部１１１０は、質問生成部９２０が生成した自然文（改訂質問ＲＱ）と、入力質問Ｑとの誤差を算出すると共に、質問応答実行部９１０による入力質問Ｑに対する回答と、当該入力質問Ｑに対する回答の正解との誤差を算出する。そして、これらの誤差を用いて、任意の最適化方法により改訂質問生成モデルのパラメータ（学習済みでない改訂質問生成モデルパラメータ）を更新する。パラメータ更新部１１１０によりパラメータが更新されることで、改訂質問生成モデルが学習される。 Here, the revised question generation model learning unit 1100 includes a question response execution unit 910, a question generation unit 920, and a parameter update unit 1110. The question response execution unit 910 and the question generation unit 920 are as described above. The parameter update unit 1110 calculates an error between the natural sentence (revised question RQ) generated by the question generation unit 920 and the input question Q, and answers to the input question Q by the question response execution unit 910 and the input question Q Calculate the error from the correct answer to. Then, using these errors, the parameters of the revised question generation model (revised question generation model parameters that have not been learned) are updated by an arbitrary optimization method. The revised parameter generation model is learned by updating the parameters by the parameter updating unit 1110.
　（質問生成装置１００のハードウェア構成）
　本発明の第二の実施の形態における質問生成装置１００のハードウェア構成は、第一の実施形態と同様とすれば良いため、その説明を省略する。 (Hardware configuration of question generation device 100)
The hardware configuration of the question generation device 100 according to the second embodiment of the present invention may be the same as that of the first embodiment, and a description thereof will be omitted.
　（改訂質問の生成処理）
　次に、本発明の第二の実施形態における改訂質問の生成処理について、図１５を参照しながら説明する。図１５は、本発明の第二の実施形態における改訂質問の生成処理の一例を示すフローチャートである。なお、改訂質問の生成処理では、改訂質問生成部９００を実現する改訂質問生成モデルは、ニューラルネットワークで実現されており、かつ、学習済みであるものとする。 (Revision question generation process)
Next, the revision question generation process in the second embodiment of the present invention will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of a revision question generation process according to the second embodiment of the present invention. In the revised question generation process, it is assumed that the revised question generation model for realizing the revised question generation unit 900 is realized by a neural network and has been learned.
　ここで、本発明の第二の実施形態における改訂質問生成部９００を実現する改訂質問生成モデルの一例を図１６に示す。図１６に示すように、本発明の第二の実施形態では、改訂質問生成モデルは、文書エンコード層、質問エンコード層、文書・質問照合層、機械読解モデリング層、機械読解出力層、回答ベクトル生成層、デコード層、及び改訂質問単語生成層で構成されるニューラルネットワークである。これらの層のうち、文書エンコード層、質問エンコード層、文書・質問照合層、機械読解モデリング層、及び機械読解出力層によって質問応答実行部９１０が実現される。また、回答ベクトル生成層、デコード層、及び改訂質問単語生成層によって質問生成部９２０が実現される。 Here, an example of a revised question generation model for realizing the revised question generation unit 900 in the second embodiment of the present invention is shown in FIG. As shown in FIG. 16, in the second embodiment of the present invention, the revised question generation model includes a document encoding layer, a question encoding layer, a document / question matching layer, a machine reading modeling layer, a machine reading output layer, and an answer vector generation. It is a neural network composed of a layer, a decode layer, and a revised question word generation layer. Of these layers, the question response execution unit 910 is realized by the document encoding layer, the question encoding layer, the document / question matching layer, the machine reading modeling layer, and the machine reading output layer. The question generation unit 920 is realized by the answer vector generation layer, the decode layer, and the revised question word generation layer.
　なお、文書エンコード層、質問エンコード層、文書・質問照合層、及び機械読解モデリング層は、第一の実施形態における照合部２１０に相当する。また、デコード層及び改訂質問単語生成層は、第一の実施形態における質問復元部２２０に相当する。 The document encoding layer, the question encoding layer, the document / question matching layer, and the machine reading modeling layer correspond to the matching unit 210 in the first embodiment. The decode layer and the revised question word generation layer correspond to the question restoration unit 220 in the first embodiment.
　本発明の第二の実施形態における訂正質問生成モデルを実現するニューラルネットワークは、ニューラルネットワークで自然文を生成するための手法であるEncoder-Decoderモデルと、ニューラルネットワークで質問応答の回答を生成する機械読解モデルとをベースに構成されている。機械読解モデルでは、回答候補となる記述を関連文書Ｘ中から直接抜き出す（つまり、記述を抜き出す際の始点及び終点の位置を推定する）ことにより、回答候補の生成を実現する。この機械読解モデルは、文書・質問照合層、機械読解モデリング層及び機械読解出力層で構成される。なお、Encoder-Decoderモデルの詳細については、例えば、上記の参考文献１を参照されたい。また、機械読解モデルの詳細については、例えば、上記の非特許文献１を参照されたい。 The neural network that realizes the corrected question generation model in the second embodiment of the present invention includes an encoder-decoder model that is a method for generating a natural sentence in the neural network, and a machine that generates an answer to the question response in the neural network. It is based on a reading model. In the machine reading model, the description of the answer candidate is directly extracted from the related document X (that is, the position of the start point and the end point when the description is extracted is estimated), thereby generating the answer candidate. This machine reading model is composed of a document / question matching layer, a machine reading modeling layer, and a machine reading output layer. For details of the Encoder-Decoder model, see, for example, Reference Document 1 above. For details of the machine reading model, see Non-Patent Document 1 above, for example.
　以降の改訂質問の生成処理では、図１６に示す改訂質問生成モデルを参照も参照しながら、各層の詳細な処理についても説明する。 In the subsequent revision question generation processing, detailed processing of each layer will be described with reference to the revision question generation model shown in FIG.
　ステップＳ３０１：テキスト処理部８００は、自然文で記述された入力質問と関連文書とを入力する。 Step S301: The text processing unit 800 inputs an input question described in a natural sentence and a related document.
　ステップＳ３０２：テキスト処理部８００は、入力した入力質問及び関連文書をそれぞれ単語系列に変換する。上述したように、以降では、入力質問がＪ個の単語トークンの単語系列Ｑ、関連文書がＴ個の単語トークンの単語系列Ｘにそれぞれ変換されたものとして、「入力質問Ｑ」及び「関連文書Ｘ」と表す。 Step S302: The text processing unit 800 converts each input question and related document into a word series. As described above, hereinafter, it is assumed that the input question is converted to the word sequence Q of J word tokens and the related document is converted to the word sequence X of T word tokens. X ".
　なお、単語系列で表された入力質問Ｑ及び関連文書Ｘが質問生成装置１００に入力された場合は、上記のステップＳ３０２は行われなくても良い。 In addition, when the input question Q and the related document X represented by the word series are input to the question generation device 100, the above step S302 may not be performed.
　ステップＳ３０３：改訂質問生成部９００は、以下のステップＳ３０３－１～ステップＳ３０３－３により、マッチング情報として、デコード層の初期状態とする状態ベクトルｈｑ０及びｈＭ０を生成する。 Step S303: revised question generator 900, the following steps S303-1 ~ step S303-3, as the matching information to generate a state vector h q0 and h M0 to the initial state of the decoding layer.
　ステップＳ３０３－１：まず、改訂質問生成部９００の質問応答実行部９１０は、関連文書Ｘ及び入力質問Ｑを入力し、図１６に示す改訂質問生成モデルの文書エンコード層及び質問エンコード層の処理として、関連文書Ｘ及び入力質問Ｑをそれぞれｄ次元の単語ベクトル系列に変換（エンコード）する。すなわち、質問応答実行部９１０は、関連文書Ｘ及び入力質問Ｑをそれぞれ構成する各単語トークンをｄ次元の実ベクトル化して単語ベクトル系列を作成する。 Step S303-1: First, the question response execution unit 910 of the revised question generation unit 900 inputs the related document X and the input question Q, and performs processing of the document encoding layer and the question encoding layer of the revised question generation model shown in FIG. The related document X and the input question Q are each converted (encoded) into a d-dimensional word vector sequence. That is, the question response execution unit 910 creates a word vector sequence by converting each word token constituting the related document X and the input question Q into a d-dimensional real vector.
　また、質問応答実行部９１０は、入力質問Ｑをｄ次元の単語ベクトル系列にエンコードした際の状態ベクトルｈｑ０を出力する。 Further, the question response execution unit 910 outputs a state vector h q0 when the input question Q is encoded into a d-dimensional word vector sequence.
　なお、本発明の第二の実施形態では、関連文書Ｘの単語ベクトル系列をＨで表すものとして、「文書ベクトル系列Ｈ」と表す。また、入力質問Ｑの単語ベクトル系列はＵで表すものとして、「質問ベクトル系列Ｕ」と表す。このとき、文書ベクトル系列はＨ∈Ｒｄ×Ｔであり、質問ベクトル系列がＵ∈Ｒｄ×Ｊである。 In the second embodiment of the present invention, the word vector series of the related document X is expressed as “document vector series H” as H. In addition, the word vector series of the input question Q is expressed as “question vector series U” as represented by U. At this time, the document vector sequence is HεR d × T , and the query vector sequence is UεR d × J.
　ここで、関連文書Ｘ及び入力質問Ｑをそれぞれｄ次元の単語ベクトル系列にエンコードするための手法については、文書ベクトル系列及び質問ベクトル系列が生成できれば任意の手法を採用することができる。例えば、関連文書Ｘと入力質問Ｑとをそれぞれ単語埋め込み層（Word Embedding Layer）に入力して各単語トークンをｄ次元の実ベクトルに変換した後に、ＲＮＮによって単語ベクトル系列に変換する手法を用いることができる。この他にも、例えば、注意機構（attention）を用いたエンコードを行っても良い。ただし、デコード層（Decode Layer）では、質問エンコード層から出力された状態ベクトルｈｑ０を初期状態として用いるため、任意の方法で状態ベクトルｈｑ０を生成しておく必要がある。 Here, as a method for encoding the related document X and the input question Q into d-dimensional word vector sequences, any method can be adopted as long as the document vector sequence and the question vector sequence can be generated. For example, a method of inputting a related document X and an input question Q to a word embedding layer (Word Embedding Layer) and converting each word token into a d-dimensional real vector, and then converting the word token into a word vector sequence by RNN is used. Can do. In addition to this, for example, encoding using an attention mechanism (attention) may be performed. However, since the decoding layer (Decode Layer) uses the state vector h q0 output from the question encoding layer as an initial state, it is necessary to generate the state vector h q0 by an arbitrary method.
　なお、本発明の第二の実施形態では、質問エンコード層でのみ状態ベクトルｈｑ０を生成する場合について説明するが、文書エンコード層でのみ又は文書エンコード層でも状態ベクトルｈｘ０が生成されても良い。文書エンコード層でのみ状態ベクトルｈｘ０が生成された場合には、デコード層では、状態ベクトルｈｘ０を初期状態として用いれば良い。一方で、文書エンコード層及び質問エンコード層で状態ベクトルｈｑ０及び状態ベクトルｈｘ０がそれぞれ生成された場合には、デコード層では、これらの状態のベクトルのうちの一方又は両方を初期状態として用いることができる。 In the second embodiment of the present invention, the case where the state vector h q0 is generated only in the question encoding layer will be described. However, the state vector h x0 may be generated only in the document encoding layer or in the document encoding layer. . When the state vector h x0 is generated only in the document encoding layer, the decoding layer may use the state vector h x0 as an initial state. On the other hand, when the state vector h q0 and the state vector h x0 are respectively generated in the document encoding layer and the question encoding layer, the decoding layer uses one or both of these state vectors as the initial state. Can do.
　ステップＳ３０３－２：次に、改訂質問生成部９００の質問応答実行部９１０は、図１６に示す改訂質問生成モデルの文書・質問照合層の処理として、文書ベクトル系列Ｈ及び質問ベクトル系列Ｕを用いて、機械読解を行うために関連文書Ｘ中で入力質問Ｑと関連する情報を発見及び抽出する。この発見及び抽出は、関連文書Ｘと入力質問Ｑとを照合することで行われる。 Step S303-2: Next, the question response execution unit 910 of the revised question generation unit 900 uses the document vector series H and the question vector series U as processing of the document / question matching layer of the revised question generation model shown in FIG. Thus, information related to the input question Q is found and extracted in the related document X for machine reading. This discovery and extraction is performed by collating the related document X with the input question Q.
　ここで、関連文書Ｘと入力質問Ｑとを照合する方法としては、任意の手法を採用することができる。例えば、注意機構（attention）を用いたBiDAFを採用することができる。また、例えば、ＣＮＮ（Convolutional Neural Network）を使用したＱＡＮｅｔを採用することもできる。なお、注意機構（attention）を用いたBiDAFの詳細については、例えば、上記の非特許文献１を参照されたい。また、ＣＮＮを使用したＱＡＮｅｔの詳細については、例えば、以下の参考文献７を参照されたい。 Here, as a method of collating the related document X with the input question Q, any method can be adopted. For example, BiDAF using an attention mechanism can be employed. Also, for example, QANet using CNN (Convolutional Neural Network) can be adopted. For details of BiDAF using an attention mechanism, see Non-Patent Document 1 above, for example. For details of QANet using CNN, refer to Reference Document 7 below, for example.
　［参考文献７］
　Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. Le. QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension. ICLR2018
　これにより、関連文書Ｘと入力質問Ｑとの照合結果として、ｒ次元の実ベクトル系列である照合ベクトル系列Ｇ∈Ｒｒ×Ｔが出力される。ここで、ｒは、関連文書Ｘと入力質問Ｑとの照合に用いる手法によって異なる。なお、この照合ベクトル系列Ｇは、第一の実施形態におけるアテンション行列Ｇに相当する。 [Reference 7]
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. Le.QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension.ICLR2018
As a result, a collation vector sequence GεR r × T, which is an r-dimensional real vector sequence, is output as a collation result between the related document X and the input question Q. Here, r differs depending on the method used for collation between the related document X and the input question Q. This matching vector series G corresponds to the attention matrix G in the first embodiment.
　ステップＳ３０３－３：改訂質問生成部９００の質問応答実行部９１０は、図１６に示す改訂質問生成モデルの機械読解モデリング層の処理として、照合ベクトル系列Ｇを用いて、機械読解モデリングベクトル系列Ｍ∈Ｒｄ×Ｔを作成する。ここで、機械読解モデリングベクトル系列Ｍは、例えば、文書エンコード層及び質問エンコード層と同様に、照合ベクトル系列Ｇに対してＲＮＮを用いた手法を行うことで、機械読解モデリングベクトル系列Ｍを作成する。また、このとき、質問応答実行部９１０は、質問エンコード層と同様に、隠れ状態ベクトルｈＭ０を生成する。この隠れ状態ベクトルｈＭ０は、デコード層の初期状態として用いられる。なお、機械読解モデリングベクトル系列Ｍは、第一の実施形態におけるマッチング行列Ｍに相当する。 Step S303-3: The question answer execution unit 910 of the revised question generation unit 900 uses the matching vector sequence G as a machine reading modeling layer process of the revised question generation model shown in FIG. R d × T is created. Here, the machine-reading modeling vector sequence M is created by performing a technique using RNN on the collation vector sequence G, for example, as in the document encoding layer and the question encoding layer. . At this time, the question response execution unit 910 generates a hidden state vector h M0 in the same manner as the question encoding layer. This hidden state vector h M0 is used as the initial state of the decode layer. The machine reading modeling vector series M corresponds to the matching matrix M in the first embodiment.
　ステップＳ３０４：次に、改訂質問生成部９００の質問応答実行部９１０は、図１６に示す改訂質問生成モデルの機械読解出力層の処理として、機械読解モデリングベクトル系列Ｍを用いて、回答候補を生成する。この回答候補の生成は、関連文書Ｘ中から回答候補となる記述の始点及び終点を抽出することにより行われる。 Step S304: Next, the question answer execution unit 910 of the revised question generation unit 900 generates answer candidates by using the machine reading modeling vector sequence M as the process of the machine reading output layer of the revised question generation model shown in FIG. To do. The generation of the answer candidates is performed by extracting the start point and the end point of the description as the answer candidate from the related document X.
　ここで、始点に関しては、図１６に示す改訂質問生成モデルの機械読解出力層に含まれる回答始点出力層の処理として、機械読解モデリングベクトル系列Ｍを重みＷ０∈Ｒ１×ｄにより線形変換することで始点ベクトルＯｓｔａｒｔ∈ＲＴを作成した上で、この始点ベクトルＯｓｔａｒｔに対して系列長Ｔでsoftmax関数を適用して確率分布Ｐｓｔａｒｔに変換する。そして、この確率分布Ｐｓｔａｒｔを用いて、最も確率が高いｔｓｔａｒｔ（０≦ｔｓｔａｒｔ≦Ｔ）番目の要素を関連文書Ｘから抜き出して、始点の単語とする。 Here, with respect to the start point, as the processing of the answer start point output layer included in the machine reading output layer of the revised question generation model shown in FIG. 16, the machine reading modeling vector sequence M is linearly converted with weights W 0 ∈R 1 × d. after having created the starting point vector O start ∈R T by, it converted into a probability distribution P start by applying the softmax function sequence length T with respect to the starting point vector O start. Then, using this probability distribution P start , the t start (0 ≦ t start ≦ T) -th element with the highest probability is extracted from the related document X and used as the starting word.
　一方で、終点に関しては、図１６に示す改訂質問生成モデルの機械読解出力層に含まれる回答終点出力層の処理として、まず始点ベクトルＯｓｔａｒｔ及び機械読解モデリングベクトル系列ＭをＲＮＮに入力して新しい機械読解モデリングベクトル系列Ｍ´を作成する。その後、始点と同様の方法により新しい機械読解モデリングベクトル系列Ｍ´から確率分布Ｐｅｎｄを得て、この確率分布Ｐｅｎｄを用いて、最も確率が高いｔｅｎｄ（ｔｓｔａｒｔ≦ｔｅｎｄ≦Ｔ）番目の要素を関連文書Ｘから抜き出して、終点の単語とする。 On the other hand, regarding the end point, as the processing of the answer end point output layer included in the machine reading output layer of the revised question generation model shown in FIG. 16, first, the start point vector O start and the machine reading modeling vector sequence M are input to the RNN and new. A machine reading modeling vector series M ′ is created. After that, a probability distribution P end is obtained from the new machine-reading modeling vector sequence M ′ by the same method as the starting point, and t end (t start ≦ t end ≦ T) th with the highest probability is obtained using this probability distribution P end. Are extracted from the related document X and set as the end point word.
　これにより、関連文書Ｘ中のｔｓｔａｒｔ番目（始点）の単語からｔｅｎｄ番目（終点）の単語までの区間が回答（回答候補）として抽出される。 Thereby, a section from the t start- th (start point) word to the t end- th (end point) word in the related document X is extracted as an answer (answer candidate).
　Ｎ個の回答（回答候補）を抽出するには、まず、Ｐｓｔａｒｔ及びＰｅｎｄを用いて、Ｐ（ｉ，ｋ）＝Ｐｓｔａｒｔ（ｉ）×Ｐｅｎｄ（ｋ）を計算する。ただし、０≦ｉ≦Ｔ、かつ、ｉ≦ｋ≦Ｔである。そして、Ｐ（ｉ，ｋ）が上位Ｎ個のｉ，ｋの組み合わせを始点及び終点とすれば良い。これにより、上位Ｎ個のｉ，ｋの組み合わせに対応する区間が、Ｎ個の回答（回答候補）としてそれぞれ抽出される。 In order to extract N answers (answer candidates), P (i, k) = P start (i) × P end (k) is first calculated using P start and P end . However, 0 ≦ i ≦ T and i ≦ k ≦ T. Then, a combination of i, k with the top N P (i, k) may be used as the start point and the end point. As a result, the sections corresponding to the top N i, k combinations are respectively extracted as N answers (answer candidates).
　なお、質問応答実行部９１０は、Ｎ個の回答（回答候補）のそれぞれの始点及び終点を出力しても良いし、Ｎ個の回答（回答候補）そのものを出力しても良いし、Ｎ個の回答（回答候補）のそれぞれの始点の単語及び終点の単語を出力しても良い。本発明の第二の実施形態では、Ｎ個の回答（回答候補）のそれぞれの始点及び終点が出力されるものとする。また、以降のステップＳ３０５は、Ｎ個の始点及び終点の組のそれぞれに対して実行されるが、以降では、或る１組の始点ｔｓｔａｒｔ及び終点ｔｅｎｄを「回答候補Ａ」として、この回答候補Ａに関してステップＳ３０５を説明する。 The question answer execution unit 910 may output the start point and the end point of each of N answers (answer candidates), may output the N answers (answer candidates) themselves, or N The start point word and end point word of each answer (answer candidate) may be output. In the second embodiment of the present invention, it is assumed that the start point and the end point of each of N answers (answer candidates) are output. Further, the subsequent step S305 is executed for each of the N start point and end point sets. Hereinafter, a certain set of start point t start and end point t end is set as “answer candidate A”. Step S305 will be described for the answer candidate A.
　ステップＳ３０５：改訂質問生成部９００は、以下のステップＳ３０５－１～ステップＳ３０５－３により、回答候補Ａに対応する改訂質問を生成する。 Step S305: The revised question generation unit 900 generates a revised question corresponding to the answer candidate A through the following steps S305-1 to S305-3.
　ステップＳ３０５－１：改訂質問生成部９００の質問生成部９２０は、回答候補Ａ（つまり、始点ｔｓｔａｒｔ及び終点ｔｅｎｄ）を入力し、図１６に示す改訂質問生成モデルの回答ベクトル生成層の処理として、回答候補Ａに対応する回答ベクトル Step S305-1: The question generation unit 920 of the revised question generation unit 900 inputs the answer candidate A (that is, the start point t start and the end point t end ), and processes the response vector generation layer of the revised question generation model shown in FIG. As an answer vector corresponding to answer candidate A
Figure JPOXMLDOC01-appb-M000019
 を作成する。ここで、ｄａは回答ベクトルの次元数を表す。
Figure JPOXMLDOC01-appb-M000019
 Create Here, d a represents the number of dimensions of the answer vector.
　回答ベクトルａの作成方法は、入力として回答候補Ａ（つまり、始点ｔｓｔａｒｔ及び終点ｔｅｎｄ）を用いて、回答ベクトルａを作成することができるものであれば任意の方法を採用することができる。例えば、始点ｔｓｔａｒｔから終点ｔｅｎｄまでの区間の記述を一度単語系列に変換した上で、この単語系列を文書エンコード層によってベクトルに変換したもの回答ベクトルａとしても良いし、始点ｔｓｔａｒｔ及び終点ｔｅｎｄで決定される区間Ｈ（ｔｓｔａｒｔ，ｔｅｎｄ）∈Ｒｄ×ｌ（ｌは回答候補Ａの系列長）を文書ベクトル系列から抽出し、抽出した区間に対応するベクトル系列に対してＲＮＮを適用したり、重心ベクトルを計算したりする等により回答ベクトルａを作成しても良い。 As a method for creating the answer vector a, any method can be adopted as long as the answer vector a can be created using the answer candidate A (that is, the start point t start and the end point t end ) as an input. . For example, the description of the section from the start point t start to the end point t end is once converted into a word sequence, and this word sequence is converted into a vector by the document encoding layer, and the answer vector a may be used. The start point t start and the end point section H (t start, t end) which is determined by t end the RNN respect ∈R d × l (l is the sequence length of the answer candidate a) vector sequences was extracted from the document vector sequence, corresponding to the extracted sections Alternatively, the answer vector a may be created by applying or calculating the center of gravity vector.
　なお、例えば、関連文書Ｘ中の記述をそのまま抽出したものを回答（回答候補Ａ）とするのではなく、関連文書Ｘ中の記述を参考に回答（回答候補Ａ）となる文を生成する方法を用いた場合、生成された文（回答となる文）を入力とし、回答ベクトル生成層の処理として、回答ベクトルａを作成すれば良い。 Note that, for example, a method for generating a sentence to be an answer (answer candidate A) with reference to the description in the related document X, instead of using the answer (answer candidate A) extracted as it is in the related document X Is used, the generated sentence (the sentence that becomes the answer) is used as an input, and the answer vector a may be created as the process of the answer vector generation layer.
　ステップＳ３０５－２：改訂質問生成部９００の質問生成部９２０は、図１６に示す改訂質問生成モデルのデコード層の処理として、ＲＮＮにより、回答ベクトルａを用いて、改訂質問を構成する単語を出力するためのベクトルを作成する。ここで、このＲＮＮでは、状態ベクトルの初期値（初期状態）として、質問応答実行部９１０から出力された状態ベクトルｈｑ０及びｈＭ０を用いる。 Step S305-2: The question generation unit 920 of the revised question generation unit 900 outputs the words constituting the revised question using the answer vector a by the RNN as processing of the decoding layer of the revised question generation model shown in FIG. Create a vector to do. Here, in this RNN, the state vectors h q0 and h M0 output from the question response execution unit 910 are used as initial values (initial states) of the state vectors.
　上記の状態ベクトルｈｑ０及びｈＭ０の使用方法については、任意の方法を採用することができる。例えば、ＲＮＮを２層にして、１層目のＲＮＮの初期状態をｈｑ０、２層目のＲＮＮの初期状態をｈＭ０としても良い。又は、例えば、１層のＲＮＮで使用する場合には、次元数を整合させるために線形変換を行った上で、２つの状態ベクトルｈｑ０及びｈＭ０の平均ベクトルを初期状態としても良いし、２つの状態ベクトルｈｑ０及びｈＭ０のいずれか一方のみを初期状態としても良い。 An arbitrary method can be adopted as a method of using the state vectors h q0 and h M0 . For example, the RNN may be two layers, and the initial state of the first layer RNN may be h q0 , and the initial state of the second layer RNN may be h M0 . Or, for example, when used in a one-layer RNN, after performing linear transformation to match the number of dimensions, an average vector of two state vectors h q0 and h M0 may be set as an initial state, Only one of the two state vectors h q0 and h M0 may be set as the initial state.
　また、状態ベクトルｈＭ０の代わりに、文書エンコード層の状態ベクトルｈｘ０を用いて、状態ベクトルｈｑ０及びｈｘ０をデコード層の初期状態を決定しても良い。これにより、例えば、Ｐ（ｉ，ｋ）が同程度の異なる回答候補が複数存在するような場合（つまり、質問内容が曖昧であるような場合等）に、回答精度の向上が期待できる。 Further, instead of the state vector h M0 , the state vector h x0 of the document encoding layer may be used to determine the initial state of the decoding layer for the state vectors h q0 and h x0 . Thereby, for example, when there are a plurality of different answer candidates having the same P (i, k) (that is, when the question content is ambiguous, etc.), it is expected to improve the answer accuracy.
　ここで、Encoder-Decoderモデルでは、デコード層には１つ前に生成した単語の埋め込みベクトル Here, in the Encoder-Decoder model, the decoding layer contains the embedded vector of the previously generated word.
Figure JPOXMLDOC01-appb-M000020
 を入力する。ここで、ｄｅは単語埋め込みベクトルの次元数を表す。これに対して、本発明の第二の実施形態では、単語埋め込みベクトルに回答ベクトルを結合したベクトル
Figure JPOXMLDOC01-appb-M000020
 Enter. Here, d e represents the number of dimensions of the word embedding vectors. In contrast, in the second embodiment of the present invention, a vector in which a response vector is combined with a word embedding vector.
Figure JPOXMLDOC01-appb-M000021
 をデコード層に入力するものとする。なお、状態ベクトルの初期値と入力されるベクトル以外は、Encoder-Decoderモデルのデコード層と同様である。したがって、例えば、注意機構（attention）やコピー等、Encoder-Decoderモデルのデコード層で用いられる任意の手法を、図１６に示す改訂質問生成モデルのデコード層に適用しても良い。
Figure JPOXMLDOC01-appb-M000021
 Are input to the decode layer. Note that, except for the initial value of the state vector and the input vector, it is the same as the decoding layer of the Encoder-Decoder model. Therefore, for example, any technique used in the decoding layer of the Encoder-Decoder model, such as attention mechanism or copying, may be applied to the decoding layer of the revised question generation model shown in FIG.
　ステップＳ３０５－３：改訂質問生成部９００の質問生成部９２０は、Encoder-Decoderモデルと同様に、デコード層の出力から改訂質問を構成するｓ番目の単語ｙｓを生成する。すなわち、例えば、デコード層の出力結果を線形変換した後に、softmax関数により関連文書Ｘ中の単語の生成確率を生成する。そして、例えば、単語の生成確率が最大となる単語を、ｓ番目の単語ｙｓとして生成する。これを単語ｙｓとして＜ＥＯＳ＞が生成されるまで繰り返すことで、回答候補Ａに対応する改訂候補を構成する単語が生成される。なお、ｙ０は＜ＢＯＳ＞であるものとする。 Step S305-3: The question generation unit 920 of the revision question generation unit 900 generates the s-th word y s constituting the revision question from the output of the decode layer, similarly to the Encoder-Decoder model. That is, for example, after linearly converting the output result of the decoding layer, the word generation probability in the related document X is generated by the softmax function. Then, for example, words, word generation probability becomes the maximum, is generated as s-th word y s. By repeating this until the word y s is <EOS> is generated, the words constituting the revised candidate corresponding to the answer candidate A is generated. It should be noted, y 0 is assumed to be <BOS>.
　ステップＳ３０６：最後に、出力部１０００は、Ｎ個の回答（回答候補）と、これらのＮ個の回答のそれぞれに対応するＮ個の改訂質問ＲＱとを出力する。 Step S306: Finally, the output unit 1000 outputs N answers (answer candidates) and N revised questions RQ corresponding to each of these N answers.
　（改訂質問生成モデルの学習処理）
　次に、本発明の第二の実施形態における改訂質問生成モデルの学習処理について、図１７を参照しながら説明する。図１７は、本発明の第二の実施形態における改訂質問生成モデルの学習処理の一例を示すフローチャートである。ここで、本発明の第二の実施形態では、改訂質問生成モデルを学習するために、機械読解のコーパスを用いるものとする。機械読解のコーパスは、「質問」と、「質問対象となる文書」と、「質問対象となる文書中の回答範囲（又は、当該回答範囲の文字列）」との組が複数含まれる。このとき、コーパスに含まれる「質問対象となる文書」を関連文書Ｘ、コーパスに含まれる「質問」を入力質問Ｑとし、当該入力質問Ｑに対する回答の正解Ａｔｒｕｅは、当該コーパス中の「質問対象となる文書中の回答範囲（又は、当該回答範囲の文字列）」をそのまま使用するものとする。そして、入力質問Ｑと、この入力質問Ｑに対する回答の正解Ａｔｒｕｅとを質問応答実行部９１０における機械読解処理のための学習データとする。なお、本発明の第二の実施の形態では、回答の正解Ａｔｒｕｅは、始点及び終点の組で表されているものとする。 (Learning process of revised question generation model)
Next, the revised question generation model learning process in the second embodiment of the present invention will be described with reference to FIG. FIG. 17 is a flowchart showing an example of the learning process of the revised question generation model in the second embodiment of the present invention. Here, in the second embodiment of the present invention, a machine-reading corpus is used to learn the revised question generation model. The machine-reading corpus includes a plurality of sets of “question”, “document to be questioned”, and “answer range (or character string of the answer range) in the question target document”. At this time, the “document to be questioned” included in the corpus is the related document X, the “question” included in the corpus is the input question Q, and the correct answer A true for the input question Q is the “question” in the corpus. The response range (or character string of the response range) in the target document is used as it is. Then, the input question Q and the correct answer A true of the answer to the input question Q are used as learning data for machine reading processing in the question response execution unit 910. In the second embodiment of the present invention, it is assumed that the correct answer A true of the answer is represented by a set of a start point and an end point.
　ステップＳ４０１：テキスト処理部８００は、複数の学習データ（すなわち、学習データセット）と、関連文書とを入力する。 Step S401: The text processing unit 800 inputs a plurality of learning data (that is, a learning data set) and related documents.
　ステップＳ４０２：テキスト処理部８００は、入力した複数の学習データにそれぞれ含まれる複数の入力質問と、関連文書とを、単語系列である複数の入力質問Ｑと、関連文書Ｘとにそれぞれ変換する。ただし、機械読解のコーパスを用いる場合、入力された複数の入力質問及び関連文書は、既に単語系列で表現されていることが多いため、このステップＳ４０２は行わなくても良い。 Step S402: The text processing unit 800 converts a plurality of input questions and related documents respectively included in the plurality of input learning data into a plurality of input questions Q and a related document X that are word sequences. However, in the case of using a machine-reading corpus, a plurality of input questions and related documents that have been input are often already expressed in a word sequence, and therefore this step S402 need not be performed.
　なお、改訂質問生成モデルの学習処理は、例えば、学習データセットを所定の個数のミニバッチに分割した上で、ミニバッチ毎に、改訂質問生成モデルのパラメータを更新する。 In the revised question generation model learning process, for example, the learning data set is divided into a predetermined number of mini-batches, and the parameters of the revised question generation model are updated for each mini-batch.
　以下のステップＳ４０３～ステップＳ４０６は、ミニバッチに含まれる各学習データを用いて繰り返し実行される。一方で、以下のステップＳ４０７～ステップＳ４０９は、ミニバッチに含まれる全ての学習データに対してステップ４０１～ステップＳ２０６が実行された後に実行される。 The following steps S403 to S406 are repeatedly executed using each learning data included in the mini-batch. On the other hand, the following steps S407 to S409 are executed after steps 401 to S206 are executed for all the learning data included in the mini-batch.
　ステップＳ４０３：欠損質問作成部３００は、学習データである入力質問Ｑの一部を欠損させた質問Ｑ（欠損質問Ｑ）を作成する。なお、当該入力質問Ｑは欠損質問Ｑに対する正解データとなるため、以降では、入力質問Ｑを正解質問Ｑｔｒｕｅと表す。 Step S403: The missing question creation unit 300 creates a question Q (missing question Q) in which a part of the input question Q that is learning data is missing. Since the input question Q is correct answer data for the missing question Q, the input question Q is hereinafter referred to as a correct question Q true .
　ここで、欠損質問Ｑの作成方法としては、任意の手法を作成することができる。例えば、学習済みのEncoder-Decoderモデルを用いて統計的に欠損質問Ｑを作成しても良いし、文の係り受け等の構文情報を用いて文節や句を切り落とすことで欠損質問Ｑを作成しても良い。又は、自然言語処理のタスクの一つである文圧縮の手法を用いて欠損質問Ｑを作成しても良い。 Here, as a method for creating the missing question Q, any method can be created. For example, the missing question Q may be statistically created using the trained Encoder-Decoder model, or the missing question Q is created by cutting off clauses and phrases using syntax information such as sentence dependency. May be. Alternatively, the missing question Q may be created using a sentence compression technique that is one of the tasks of natural language processing.
　ステップＳ４０４：改訂質問生成モデル学習部１１００の質問応答実行部９１０は、マッチング情報を生成する。このステップＳ４０４は、図１５のステップＳ３０３における入力質問Ｑを欠損質問Ｑと読み替えることで、ステップＳ３０３と同様であるため、その説明を省略する。 Step S404: The question response execution unit 910 of the revised question generation model learning unit 1100 generates matching information. Since this step S404 is the same as step S303 by replacing the input question Q in step S303 of FIG. 15 with the missing question Q, the description thereof is omitted.
　ステップＳ４０５：改訂質問生成モデル学習部１１００の質問応答実行部９１０は、欠損質問Ｑに対する回答候補を生成する。このステップＳ４０５は、図１５のステップＳ３０４における入力質問Ｑを欠損質問Ｑと読み替えることで、ステップＳ３０４と同様であるため、その説明を省略する。 Step S405: The question response execution unit 910 of the revised question generation model learning unit 1100 generates answer candidates for the missing question Q. This step S405 is the same as step S304 by replacing the input question Q in step S304 of FIG.
　ステップＳ４０６：改訂質問生成モデル学習部１１００の質問生成部９２０は、欠損質問Ｑの回答候補のそれぞれに対応する改訂質問ＲＱを生成する。このステップＳ４０６は、図１５のステップＳ３０５における入力質問Ｑを欠損質問Ｑと読み替えることで、ステップＳ３０５と同様であるため、その説明を省略する。 Step S406: The question generation unit 920 of the revised question generation model learning unit 1100 generates a revised question RQ corresponding to each answer candidate of the missing question Q. This step S406 is the same as step S305 by replacing the input question Q in step S305 of FIG.
　ステップＳ４０７：改訂質問生成モデル学習部１１００のパラメータ更新部１１１０は、ミニバッチに含まれる各学習データを用いてそれぞれ生成された改訂質問ＲＱと、当該学習データに含まれる入力質問Ｑ（つまり、正解質問Ｑｔｒｕｅ）の第１の誤差を計算する。また、パラメータ更新部１１１０は、ミニバッチに含まれる各学習データにそれぞれ含まれる入力質問Ｑに対する回答Ａと、当該学習データに含まれる正解Ａｔｒｕｅとの第２の誤差を計算する。ここで、回答Ａは、質問応答実行部９１０に対して入力質問Ｑ（及び関連文書Ｘ）を入力することで、質問応答における回答として得られる。 Step S407: The parameter update unit 1110 of the revised question generation model learning unit 1100 generates the revised question RQ generated using each learning data included in the mini-batch and the input question Q (that is, correct answer question) included in the learning data. Calculate the first error of Q true ). The parameter updating unit 1110 calculates a second error between the answer A to the input question Q included in each learning data included in the mini-batch and the correct answer A true included in the learning data. Here, the answer A is obtained as an answer in the question answer by inputting the input question Q (and the related document X) to the question answer execution unit 910.
　第１の誤差及び第２の誤差の計算に用いられる誤差関数としては、例えば、クロスエントロピーを用いれば良い。なお、誤差関数は、改訂質問生成モデルに応じて適宜に決定される。 For example, cross-entropy may be used as the error function used for calculating the first error and the second error. The error function is appropriately determined according to the revised question generation model.
　ステップＳ４０８：改訂質問生成モデル学習部１１００のパラメータ更新部１１１０は、上記のステップＳ４０７で計算した第１の誤差及び第２の誤差を用いて、改訂質問生成モデルのパラメータを更新する。すなわち、パラメータ更新部４１０は、例えば、上記のステップＳ４０７で計算した第１の誤差及び第２の誤差を用いて、誤差逆伝播法（バックプロパゲーション）により誤差関数の偏微分値を計算することで、改訂質問生成モデルのパラメータを更新する。これにより、改訂質問生成モデルが学習される。 Step S408: The parameter update unit 1110 of the revised question generation model learning unit 1100 updates the parameters of the revised question generation model using the first error and the second error calculated in Step S407. That is, for example, the parameter update unit 410 calculates the partial differential value of the error function by the error back propagation method (back propagation) using the first error and the second error calculated in step S407 above. The parameter of the revised question generation model is updated. Thereby, the revised question generation model is learned.
　ここで、図１６に示すように、改訂質問生成モデルがニューラルネットワークである場合、機械読解（つまり、質問応答実行部９１０）と改訂質問生成（つまり、質問生成部９２０）とのそれぞれで正解データ（つまり、改訂質問ＲＱに対する正解質問Ｑｔｒｕｅと、当該正解質問Ｑｔｒｕｅに対する正解回答Ａｔｒｕｅ）に関する誤差関数を定義し、これらの誤差関数値の和（つまり、第１の誤差と第２の誤差との和）をニューラルネットワーク全体の誤差として扱い、この誤差が小さくなるようにパラメータを更新する（すなわち、マルチタスク学習によりパラメータを更新する。）。 Here, as shown in FIG. 16, when the revised question generation model is a neural network, correct answer data is generated by each of the machine reading (that is, the question answer execution unit 910) and the revised question generation (that is, the question generation unit 920). That is, an error function is defined for the correct answer question Q true for the revised question RQ and the correct answer A true for the correct question Q true, and the sum of these error function values (ie, the first error and the second error). ) Is treated as an error of the entire neural network, and the parameter is updated so that this error is reduced (that is, the parameter is updated by multitask learning).
　（まとめ）
　以上のように、本発明の第二の実施形態における質問生成装置１００は、例えばニューラルネットワークにより実現される改訂質問生成モデルを用いて、改訂質問ＲＱの生成の前に、入力質問Ｑに対して質問応答を行って、この質問応答で得られた回答候補に対応する改訂質問ＲＱを生成する。これにより、例えば、入力質問Ｑに対する回答が一意に特定できないような場合であっても、回答候補毎に改訂質問ＲＱが生成されるため、質問応答タスクにおいて、これらの改訂質問ＲＱを用いることで、高い回答精度の実現することができるようになる。 (Summary)
As described above, the question generation device 100 according to the second embodiment of the present invention uses the revised question generation model realized by, for example, a neural network, and generates the revised question RQ with respect to the input question Q. A question answer is performed, and a revised question RQ corresponding to the answer candidate obtained by this question answer is generated. Thereby, for example, even if the answer to the input question Q cannot be uniquely identified, a revised question RQ is generated for each answer candidate, so by using these revised questions RQ in the question answering task, , It will be possible to achieve high response accuracy.
　本発明は、具体的に開示された上記の実施形態に限定されるものではなく、特許請求の範囲から逸脱することなく、種々の変形や変更が可能である。 The present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims.
　１００　　　　質問生成装置
　２００　　　　改訂質問生成部
　２１０　　　　照合部
　２２０　　　　質問復元部
　３００　　　　欠損質問作成部
　４００　　　　改訂質問生成モデル学習部 DESCRIPTION OF SYMBOLS 100 Question generation apparatus 200 Revised question generation part 210 Collation part 220 Question restoration part 300 Missing question creation part 400 Revised question generation model learning part

Claims (12)

  1. 　質問文と、該質問文に対する回答が含まれる関連文書とを入力とし、予め学習済みの機械学習モデルを用いて、前記質問文の潜在的に欠損している部分を、所定の語彙集合に含まれる単語で補った改訂質問文を生成する生成手段、
    　を有することを特徴とする質問生成装置。     Using a question sentence and a related document containing an answer to the question sentence as input, using a previously learned machine learning model, a potentially missing part of the question sentence is included in a predetermined vocabulary set Generating means for generating a revised question sentence supplemented with
    A question generating device characterized by comprising:
  2. 　前記生成手段は、
    　前記質問文に含まれる各単語と、前記関連文書に含まれる各単語との一致関係を表すマッチング情報を生成する照合手段と、
    　前記照合手段により生成されたマッチング情報を用いて、前記語彙集合の中から、前記改訂質問文を構成する各単語を生成することで、前記改訂質問文を生成する質問復元手段と、
    　を有することを特徴とする請求項１に記載の質問生成装置。     The generating means includes
    Collating means for generating matching information representing a matching relationship between each word included in the question sentence and each word included in the related document;
    Using the matching information generated by the matching means, from the vocabulary set, by generating each word constituting the revised question sentence, question restoring means for generating the revised question sentence;
    The question generation device according to claim 1, wherein
  3. 　前記質問復元手段は、
    　前記改訂質問文を構成する各単語それぞれを、前記語彙集合に含まれる単語の中から生成する第１の確率と、前記関連文書に含まれる単語の中から生成する第２の確率との加重平均によって表される第３の確率により生成する、ことを特徴とする請求項２に記載の質問生成装置。     The question restoration means includes
    A weighted average of a first probability of generating each word constituting the revised question sentence from words included in the vocabulary set and a second probability generated from words included in the related document The question generation device according to claim 2, wherein the question generation device is generated with a third probability represented by:
  4. 　前記改訂質問文は、前記質問文の潜在的に欠損している部分を、前記語彙集合に含まれる単語と前記関連文書に含まれる単語とで補った文である、ことを特徴とする請求項１乃至３の何れか一項に記載の質問生成装置。 The revised question sentence is a sentence in which a potentially missing part of the question sentence is supplemented with a word included in the vocabulary set and a word included in the related document. The question generation device according to any one of 1 to 3.
  5. 　前記生成手段は、
    　前記質問文が入力されると、前記質問文と、該質問文に対する回答が含まれる関連文書の集合とに基づき、前記集合に含まれる関連文書それぞれに対応する前記改訂質問文と、前記関連文書と前記改訂質問文との対応情報とを生成する、ことを特徴とする請求項１乃至４の何れか一項に記載の質問生成装置。     The generating means includes
    When the question sentence is input, the revised question sentence corresponding to each of the related documents included in the set based on the question sentence and a set of related documents including an answer to the question sentence, and the related document The question generation apparatus according to claim 1, wherein correspondence information between the revised question sentence and the revised question sentence is generated.
  6. 　前記生成手段は、
    　生成した改訂質問文を入力とし、該改訂質問文の潜在的に欠損している部分を補った改訂質問文を生成することを繰り返し実行する、ことを特徴とする請求項１乃至４の何れか一項に記載の質問生成装置。     The generating means includes
    5. The method according to claim 1, wherein the generated revised question text is used as an input, and the generation of a revised question text that supplements a potentially missing part of the revised question text is repeatedly executed. The question generating device according to one item.
  7. 　前記生成手段は、
    　前記質問文に対する回答候補と、該回答候補が回答に対応する前記改訂質問文とを生成する、ことを特徴とする請求項１に記載の質問生成装置。     The generating means includes
    The question generation apparatus according to claim 1, wherein an answer candidate for the question sentence and the revised question sentence corresponding to the answer are generated.
  8. 　前記生成手段は、
    　前記質問文に含まれる各単語と、前記関連文書に含まれる各単語との一致関係を表すマッチング情報を生成する照合手段と、
    　前記マッチング情報を用いて、前記回答候補を生成する機械読解手段と、
    　前記回答候補と、前記マッチング情報とを用いて、前記語彙集合の中から、前記改訂質問文を構成する各単語を生成することで、前記改訂質問文を生成する改訂質問生成手段と、
    　を有することを特徴とする請求項７に記載の質問生成装置。     The generating means includes
    Collating means for generating matching information representing a matching relationship between each word included in the question sentence and each word included in the related document;
    Machine reading means for generating the answer candidates using the matching information;
    Revision question generating means for generating the revised question sentence by generating each word constituting the revised question sentence from the vocabulary set using the answer candidate and the matching information;
    The question generation device according to claim 7, comprising:
  9. 　質問文と、該質問文に対する回答が含まれる関連文書とを入力とし、前記質問文の一部分を欠損させた欠損質問文を生成する第１の生成手段と、
    　ニューラルネットワークモデルを用いて、前記欠損質問文を、所定の語彙集合に含まれる単語で復元した復元質問文を生成する第２の生成手段と、
    　前記第２の生成手段により生成された復元質問文と、前記質問文との誤差を用いて、前記ニューラルネットワークモデルのパラメータを更新する学習手段と、
    　を有することを特徴とする質問生成装置。     First generation means for receiving a question sentence and a related document including an answer to the question sentence and generating a missing question sentence in which a part of the question sentence is missing;
    Using a neural network model, second generation means for generating a restored question sentence in which the missing question sentence is restored with a word included in a predetermined vocabulary set;
    Learning means for updating parameters of the neural network model by using an error between the restoration question sentence generated by the second generation means and the question sentence;
    A question generating device characterized by comprising:
  10. 　前記学習手段は、
    　前記質問文に対する回答の正解と、前記質問文に対する回答との誤差を更に用いて、前記ニューラルネットワークモデルのパラメータを更新する、ことを特徴とする請求項９に記載の質問生成装置。     The learning means includes
    The question generating apparatus according to claim 9, wherein the parameter of the neural network model is updated by further using an error between a correct answer to the question sentence and an answer to the question sentence.
  11. 　質問文と、該質問文に対する回答が含まれる関連文書とを入力とし、予め学習済みの機械学習モデルを用いて、前記質問文の潜在的に欠損している部分を、所定の語彙集合に含まれる単語で補った改訂質問文を生成する生成手順、
    　をコンピュータが実行することを特徴とする質問生成方法。     Using a question sentence and a related document containing an answer to the question sentence as input, using a previously learned machine learning model, a potentially missing part of the question sentence is included in a predetermined vocabulary set Generation procedure to generate a revised question sentence supplemented with
    A question generation method characterized in that a computer executes the above.
  12. 　コンピュータを、請求項１乃至１０の何れか一項に記載の質問生成装置における各手段として機能させるためのプログラム。 A program for causing a computer to function as each means in the question generation device according to any one of claims 1 to 10.
PCT/JP2019/017805 2018-06-07 2019-04-25 Question generation device, question generation method, and program WO2019235103A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/972,187 US11972365B2 (en) 2018-06-07 2019-04-25 Question responding apparatus, question responding method and program

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2018109765 2018-06-07
JP2018-109765 2018-06-07
JP2018214187A JP7087938B2 (en) 2018-06-07 2018-11-14 Question generator, question generation method and program
JP2018-214187 2018-11-14

Publications (1)

Publication Number Publication Date
WO2019235103A1 true WO2019235103A1 (en) 2019-12-12

Family

ID=68770823

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/017805 WO2019235103A1 (en) 2018-06-07 2019-04-25 Question generation device, question generation method, and program

Country Status (2)

Country Link
JP (1) JP7315065B2 (en)
WO (1) WO2019235103A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241250A (en) * 2020-01-22 2020-06-05 中国人民大学 Emotional dialogue generation system and method
CN112749539A (en) * 2020-01-20 2021-05-04 腾讯科技（深圳）有限公司 Text matching method and device, computer readable storage medium and computer equipment
WO2022003762A1 (en) * 2020-06-29 2022-01-06 日本電信電話株式会社 Question answering device, question answering method, and question answering program
JP2022544428A (en) * 2020-06-28 2022-10-19 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド Search item rewriting method, device, device and storage medium
US20220383330A1 (en) * 2021-05-27 2022-12-01 EMC IP Holding Company LLC System and method for identifying and remediating unanswered queries in application resolution reports
US11699038B2 (en) 2020-09-18 2023-07-11 Fujifilm Business Innovation Corp. Information processing apparatus

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008078670A1 (en) * 2006-12-22 2008-07-03 Nec Corporation Sentence rephrasing method, program, and system
US20160062980A1 (en) * 2014-08-29 2016-03-03 International Business Machine Corporation Question Correction and Evaluation Mechanism for a Question Answering System
JP2016045652A (en) * 2014-08-21 2016-04-04 国立研究開発法人情報通信研究機構 Enquiry sentence generation device and computer program
JP2017049681A (en) * 2015-08-31 2017-03-09 国立研究開発法人情報通信研究機構 Training device for question answering system and computer program therefor
WO2018097091A1 (en) * 2016-11-25 2018-05-31 日本電信電話株式会社 Model creation device, text search device, model creation method, text search method, data structure, and program

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3946115B2 (en) * 2002-09-19 2007-07-18 日本電信電話株式会社 Response dialogue generation method, response dialogue creation device, response dialogue creation program, and recording medium recording this program
JP4461738B2 (en) * 2003-08-13 2010-05-12 富士ゼロックス株式会社 Question answering apparatus and method
JP2006323670A (en) * 2005-05-19 2006-11-30 Nippon Telegr & Teleph Corp <Ntt> Answering method, answering device and program for questioning method
JP6007088B2 (en) * 2012-12-05 2016-10-12 Kddi株式会社 Question answering program, server and method using a large amount of comment text
CA2932401A1 (en) * 2013-12-02 2015-06-11 Qbase, LLC Systems and methods for in-memory database search

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008078670A1 (en) * 2006-12-22 2008-07-03 Nec Corporation Sentence rephrasing method, program, and system
JP2016045652A (en) * 2014-08-21 2016-04-04 国立研究開発法人情報通信研究機構 Enquiry sentence generation device and computer program
US20160062980A1 (en) * 2014-08-29 2016-03-03 International Business Machine Corporation Question Correction and Evaluation Mechanism for a Question Answering System
JP2017049681A (en) * 2015-08-31 2017-03-09 国立研究開発法人情報通信研究機構 Training device for question answering system and computer program therefor
WO2018097091A1 (en) * 2016-11-25 2018-05-31 日本電信電話株式会社 Model creation device, text search device, model creation method, text search method, data structure, and program

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112749539A (en) * 2020-01-20 2021-05-04 腾讯科技（深圳）有限公司 Text matching method and device, computer readable storage medium and computer equipment
CN112749539B (en) * 2020-01-20 2023-09-15 腾讯科技（深圳）有限公司 Text matching method, text matching device, computer readable storage medium and computer equipment
CN111241250A (en) * 2020-01-22 2020-06-05 中国人民大学 Emotional dialogue generation system and method
CN111241250B (en) * 2020-01-22 2023-10-24 中国人民大学 Emotion dialogue generation system and method
JP2022544428A (en) * 2020-06-28 2022-10-19 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド Search item rewriting method, device, device and storage medium
JP7352640B2 (en) 2020-06-28 2023-09-28 ベイジン バイドゥ ネットコム サイエンス テクノロジー カンパニー リミテッド Search item rewriting method, device, equipment and storage medium
WO2022003762A1 (en) * 2020-06-29 2022-01-06 日本電信電話株式会社 Question answering device, question answering method, and question answering program
JP7468654B2 (en) 2020-06-29 2024-04-16 日本電信電話株式会社 Question answering device, question answering method, and question answering program
US11699038B2 (en) 2020-09-18 2023-07-11 Fujifilm Business Innovation Corp. Information processing apparatus
US20220383330A1 (en) * 2021-05-27 2022-12-01 EMC IP Holding Company LLC System and method for identifying and remediating unanswered queries in application resolution reports

Also Published As

Publication number Publication date
JP2022111261A (en) 2022-07-29
JP7315065B2 (en) 2023-07-26

Similar Documents

Publication Publication Date Title
JP7087938B2 (en) Question generator, question generation method and program
WO2019235103A1 (en) Question generation device, question generation method, and program
Kamath et al. Deep learning for NLP and speech recognition
US20210124878A1 (en) On-Device Projection Neural Networks for Natural Language Understanding
Yao et al. An improved LSTM structure for natural language processing
CN110377903B (en) Sentence-level entity and relation combined extraction method
Jeong et al. Triangular-chain conditional random fields
CN110866401A (en) Chinese electronic medical record named entity identification method and system based on attention mechanism
CN107870902A (en) Neural machine translation system
EP3411835B1 (en) Augmenting neural networks with hierarchical external memory
CN110990555B (en) End-to-end retrieval type dialogue method and system and computer equipment
CN106202010A (en) The method and apparatus building Law Text syntax tree based on deep neural network
CN112925516A (en) Slot filling with context information
CN108959482A (en) Single-wheel dialogue data classification method, device and electronic equipment based on deep learning
CN113254610B (en) Multi-round conversation generation method for patent consultation
CN114676234A (en) Model training method and related equipment
JP7229345B2 (en) Sentence processing method, sentence decoding method, device, program and device
WO2020240709A1 (en) Dialog processing device, learning device, dialog processing method, learning method, and program
Dai et al. A survey on dialog management: Recent advances and challenges
JP7342971B2 (en) Dialogue processing device, learning device, dialogue processing method, learning method and program
CN111145914B (en) Method and device for determining text entity of lung cancer clinical disease seed bank
CN112183062B (en) Spoken language understanding method based on alternate decoding, electronic equipment and storage medium
CN112364659B (en) Automatic identification method and device for unsupervised semantic representation
Goutsu et al. Linguistic descriptions of human motion with generative adversarial seq2seq learning
Wang et al. Improving relation extraction by multi-task learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19814938

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19814938

Country of ref document: EP

Kind code of ref document: A1