CN112231456B - Question generation method, device, storage medium and electronic equipment - Google Patents
Question generation method, device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN112231456B CN112231456B CN202011106484.XA CN202011106484A CN112231456B CN 112231456 B CN112231456 B CN 112231456B CN 202011106484 A CN202011106484 A CN 202011106484A CN 112231456 B CN112231456 B CN 112231456B
- Authority
- CN
- China
- Prior art keywords
- target
- answer
- questions
- content
- answers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 239000013598 vector Substances 0.000 claims description 112
- 238000012549 training Methods 0.000 claims description 40
- 238000001514 detection method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000012163 sequencing technique Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The embodiment of the invention discloses a problem generation method and device. The method comprises the following steps: obtaining target answers submitted by target contents, generating a plurality of candidate questions according to the target answers and the target contents, calculating non-answer scores of answers of the candidate questions which do not exist in the target contents and answer scores of answers of the candidate questions which exist in the target contents according to the plurality of candidate questions and the target contents, eliminating the questions which cannot be answered by the target contents in the plurality of candidate questions according to the non-answer scores and the answer scores, obtaining target questions, providing the target answers and the corresponding target questions, enabling the answers to be submitted by background staff, enabling the answers to be more relevant to actual demands, eliminating the questions which cannot be answered by the answers in the target contents in the generated questions, avoiding the problem that the generated questions cannot be answered by the answers close to the actual service demands, and improving the accuracy of the questions, thereby improving the utilization rate of the generated questions.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a problem generating method, a problem generating device, a storage medium, and an electronic apparatus.
Background
The current QA (Question Answering, question answer) is mostly a decimated task, the answer is a continuous piece of text, usually a semantic concept like a named entity, the QG (Question Generation ) is a generated, the question sentence is a complete sentence, and part of the words may not appear in the document, and in many cases, the language structures of the question sentence and the answer are different, so that it can even be regarded as two different types of data. Meanwhile, QA and QG are related in probability, so that under the same framework, joint training can be realized by converting input data to deal with extraction or generation problems.
Most of the existing question generation algorithm modules are used as generation tasks to complete, and are partially combined with QA tasks to complete the generation tasks through training, but screening and verification of questions are lacking, so that the generated questions cannot be answered by answers close to actual service demands, and the accuracy of the questions is low.
Disclosure of Invention
In view of the above problems, a method, an apparatus, a storage medium, and a processor for generating a problem are provided to solve the problem that the generated problem cannot be answered with an answer close to the actual service requirement, and the accuracy of the problem is not high.
According to an aspect of the present invention, there is provided a problem generating method including:
acquiring a target answer submitted aiming at target content;
generating a plurality of candidate questions matched with the target answers according to the target answers and the target content;
calculating a no-answer score of an answer without the candidate question in the target content and an answer score of an answer with the candidate question in the target content according to the plurality of candidate questions and the target content;
according to the answer-free score and the answer-free score, eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions to obtain target questions;
and providing the target answers and the corresponding target questions.
Optionally, the obtaining the target answer submitted for the target content includes:
and receiving a target answer selected from a plurality of candidate answers in the target content, or receiving a target answer corresponding to a target position submitted for the target content.
Optionally, the generating a plurality of candidate questions matched with the target answer according to the target answer and the target content includes:
combining answer content vectors representing the target answers and the target contents according to the answer vectors of the target answers and the content vectors of the target contents;
inputting the answer content vector and the question vector of each question into the BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
and acquiring a plurality of candidate questions matched with the target answers.
Optionally, the combining the answer vector according to the target answer and the content vector of the target content to form the answer content vector representing the target answer and the target content includes:
acquiring the position information and the occurrence times of the target answers in the target content;
and forming the answer content vector by the answer vector, the content vector, the position information and the occurrence number.
Optionally, the calculating, according to the plurality of candidate questions and the target content, a no-answer score of an answer in which the candidate questions do not exist in the target content, and the answer score of the answer in which the candidate questions exist in the target content includes:
Inputting the candidate questions and target content into a RoBerta pre-training model, and calculating the no-answer score of the answer without the candidate questions in the target content and the answer score of the answer with the candidate questions in the target content.
Optionally, before the providing the target answer and the corresponding target question, the method further includes:
dividing target answers with the same stem into a group to obtain an answer question group consisting of the target answers and corresponding target questions;
and ordering the target questions from high to low in the same answer question group according to the confidence degree of the target questions.
Optionally, the providing the target answer and the corresponding target question includes:
and generating a display view of the target answer and the corresponding target question according to the answer question group.
According to another aspect of the present invention, there is provided a problem generating apparatus including:
the answer acquisition module is used for acquiring a target answer submitted aiming at the target content;
the question generation module is used for generating a plurality of candidate questions matched with the target answers according to the target answers and the target content;
a score calculating module, configured to calculate, according to the plurality of candidate questions and target content, a no-answer score of an answer in which the candidate question does not exist in the target content, and a answer score of an answer in which the candidate question exists in the target content;
The question eliminating module is used for eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions according to the answer-free score and the answer-free score to obtain target questions;
and the providing module is used for providing the target answers and the corresponding target questions.
Optionally, the answer acquisition module includes:
and the receiving sub-module is used for receiving a target answer selected from a plurality of candidate answers in the target content or receiving a target answer corresponding to a target position submitted for the target content.
Optionally, the problem generating module includes:
the vector combination sub-module is used for combining answer content vectors representing the target answers and the target content according to the answer vectors of the target answers and the content vectors of the target content;
the matching detection sub-module is used for inputting the answer content vector and the question vector of each question into the BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
and the question acquisition sub-module is used for acquiring a plurality of candidate questions matched with the target answers.
Optionally, the vector combining submodule includes:
An information acquisition unit for acquiring the position information and the occurrence number of the target answer in the target content;
and a vector composing unit for composing the answer content vector from the answer vector, the content vector, and the position information and the number of occurrences.
Optionally, the score calculating module includes:
the score calculating sub-module is used for inputting the candidate questions and the target content into the RoBerta pre-training model, calculating the unanswered scores of the answers of the candidate questions which are not present in the target content, and calculating the answered scores of the answers of the candidate questions which are present in the target content.
Optionally, the apparatus further comprises:
the grouping module is used for grouping the target answers with the same word stems into a group before the target answers and the corresponding target questions are provided, so as to obtain answer question groups consisting of the target answers and the corresponding target questions;
and the sequencing module is used for sequencing the target questions from high to low according to the confidence degrees of the target questions in the same answer question group.
Optionally, the providing module includes:
and the view generation sub-module is used for generating a display view of the target answer and the corresponding target question according to the answer question group.
According to another aspect of the present invention, there is provided a storage medium comprising a stored program, wherein the program, when run, controls a device on which the storage medium resides to perform one or more methods as described above.
According to another aspect of the present invention, there is provided an electronic apparatus including: a memory, a processor, and executable instructions stored in the memory and executable in the processor, wherein the processor implements one or more of the methods described above when executing the executable instructions.
According to the embodiment of the invention, the target answers submitted for the target content are obtained, a plurality of candidate questions are generated according to the target answers and the target content, the answer-free scores of the answers of the candidate questions which do not exist in the target content and the answer-free scores of the answers of the candidate questions which exist in the target content are calculated according to the plurality of candidate questions and the target content, the questions which cannot be answered by adopting the target content in the plurality of candidate questions are removed according to the answer-free scores and the answer-free scores, the target questions are obtained, the target answers and the corresponding target questions are provided, so that the answers are submitted by background personnel, the answers are more closely matched with actual demands, the questions which cannot be answered by the answers in the target content in the generated questions are removed, the problems which cannot be answered by the answers close to the actual service demands are avoided, and the accuracy of the questions is improved, and therefore the utilization rate of the generated questions is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a problem generation method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a problem generation method according to a second embodiment of the present invention;
fig. 3 is a block diagram of a problem generating apparatus according to a third embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Example 1
Referring to fig. 1, a flowchart of a problem generating method in the first embodiment of the present invention may specifically include:
Step 101, obtaining a target answer submitted for target content.
In an embodiment of the present invention, questions and answers are generated based on given target content, for example, in a scenario of a customer service system serving insurance customers, various documents of insurance products and services collected in advance are used as target content for generating questions and answers. The target content may include an article in text form, or a paragraph, a piece of text, etc., or any other suitable content, to which embodiments of the present invention are not limited. The target content may be text-reviewed content, for example, input content is subjected to data cleansing, unicode characters, null paragraphs, URL (uniform resource locator, uniform resource locator system) links, etc. are deleted, and background personnel may also edit and modify.
In the embodiment of the present invention, the implementation manner of personalized submitting the answer by the background personnel to obtain the target answer submitted for the target content may include various manners, for example, receiving the target answer selected from a plurality of candidate answers in the target content, or receiving the target location (such as the starting and ending locations marked in the chapter) submitted for the target content, to obtain the target answer corresponding to the target location, or any other applicable manner, which is not limited in the embodiment of the present invention.
And 102, generating a plurality of candidate questions matched with the target answers according to the target answers and the target content.
In the embodiment of the invention, the candidate questions are matched with the target answers and can be generated according to the target answers and target contents. Methods of generating questions fall into two categories, rule-based methods and neural network-based methods. The rule-based method mainly uses a group of manually created language-driven conversion rules or templates to generate problems; neural network-based approaches mainly employ a sequence-to-sequence (sequence-to-sequence) architecture of an attention mechanism to generate relevant questions for a given sentence. For example, the target answer is "20 years" and the target content includes "..once a certain company has been developed for 20 years. "is a company the year a few years? "," how much years a certain industry has progressed rapidly? ", how many years? "etc.
In the embodiment of the present invention, the implementation manner of generating the plurality of candidate questions matched with the target answer according to the target answer and the target content may include various implementation manners, for example, training a question generation module based on a BERT pretraining model, inputting the target answer and the target content into the question generation module, outputting the plurality of candidate questions matched with the target answer, or any other applicable implementation manner, which is not limited in the embodiment of the present invention.
Step 103, calculating, according to the plurality of candidate questions and the target content, a no-answer score of an answer of the candidate question which is not present in the target content, and a answer score of an answer of the candidate question which is present in the target content.
In the embodiment of the invention, some questions which cannot be answered by using the target content may exist in the generated candidate questions. In order to remove the questions which cannot be answered by using the answers in the target content, firstly, according to the candidate questions and the target content, the non-answer score of the answer without the candidate questions in the target content can be calculated, namely, the answer questions in the target content cannot be answered by using the answer scores of the answers with the candidate questions in the target content, and the answer questions in the target content can be answered by using the answers in the target content. Specific implementations include, for example, inputting the candidate questions and the target content into a RoBerta pre-training model, calculating a no-answer score for an answer to the candidate question that does not exist in the target content, and an answer score for an answer to the candidate question that exists in the target content, or any other suitable method, to which embodiments of the present invention are not limited.
And 104, eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions according to the answer-free scores and the answer-free scores to obtain target questions.
In the embodiment of the invention, for a plurality of candidate questions, answer-free scores and answer-free scores are calculated respectively, and then the questions which cannot be answered by adopting the target content in the plurality of candidate questions are removed according to the answer-free scores and the answer-free scores, so as to obtain the target questions.
In the embodiment of the invention, by comparing the answer-free score with the answer-free score, for example, whether the difference between the answer-free score and the answer-free score is greater than a preset threshold, if the difference is greater than the preset threshold, the candidate question cannot be answered by using the target content, and the candidate question needs to be removed, and if the difference is not greater than the preset threshold, the candidate question can be answered by using the target content.
For example, for three candidate questions generated: "is a company the year a few years? "," how much years a certain industry has progressed rapidly? "what years? In "three candidate questions and target content are input into the robberta pre-training model, and" how many years? "belongs to a question that cannot be answered.
And step 105, providing the target answers and the corresponding target questions.
In the embodiment of the invention, the target answers and the corresponding target questions are output as a question-answer pair and provided for an automatic question-answer system.
According to the embodiment of the invention, the target answers submitted for the target content are obtained, a plurality of candidate questions are generated according to the target answers and the target content, the answer-free scores of the answers of the candidate questions which do not exist in the target content and the answer-free scores of the answers of the candidate questions which exist in the target content are calculated according to the plurality of candidate questions and the target content, the questions which cannot be answered by adopting the target content in the plurality of candidate questions are removed according to the answer-free scores and the answer-free scores, the target questions are obtained, the target answers and the corresponding target questions are provided, so that the answers are submitted by background personnel, the answers are more closely matched with actual demands, the questions which cannot be answered by the answers in the target content in the generated questions are removed, the problems which cannot be answered by the answers close to the actual service demands are avoided, and the accuracy of the questions is improved, and therefore the utilization rate of the generated questions is improved.
Example two
Referring to fig. 2, a flowchart of a problem generating method in the second embodiment of the present invention may specifically include:
step 201, receiving a target answer selected from a plurality of candidate answers in the target content, or receiving a target answer corresponding to a target position submitted for the target content.
In the embodiment of the invention, a background person can select an answer in a candidate answer list identified by a system as a target answer, or select a target position of a self-defined answer from an article, and take a text corresponding to the target position as the target answer. Any named entity, the noun phrase that appears in a paragraph, can be the answer to the target. For example, a phrase extracted using Stanford CoreNLP (a natural language toolkit of the university of Stanford) contains a list of all named entities and nouns, and then a label for the key answer, i.e., the target answer, is selected from the list. For another example, a background person may manually select a set of spans (i.e., target locations) as target answers.
Step 202, combining answer content vectors representing the target answer and the target content according to the answer vectors of the target answer and the content vectors of the target content.
In the embodiment of the invention, a problem generation module based on a BERT pre-training model can be adopted in the problem generation. The BERT model converts each word in the text into a one-dimensional vector by querying a word vector table as a model input. In this way, the target answer is converted into a vector form and is recorded as an answer vector; the target content is converted into a vector form and recorded as a content vector. Combining the answer vector and the content vector to obtain a vector representing the target answer and the target content, and recording the vector as the answer content vector.
In an embodiment of the present invention, optionally, according to the answer vector of the target answer and the content vector of the target content, one implementation manner of combining the answer content vector representing the target answer and the target content may include: acquiring the position information and the occurrence times of the target answers in the target content; and forming the answer content vector by the answer vector, the content vector, the position information and the occurrence number.
In a long context, the target answer will often appear multiple times in the target content, causing the model to generate a question with the target answer appearing a time, which is likely to be ambiguous. Therefore, a token is designed for representing the position information and the occurrence number of the target answer in the target content, so as to solve the possible ambiguity problem of the question sentence generated by the target answer appearing at a certain time, thereby reducing the error rate of the generated problem and improving the efficiency of generating the problem.
For example, for a given context content vectorAnswer segment vector a= [ a ] 1 ，a 2 ，...，a |A| ]Wherein c |C| And a |A| Is a word vector. The vectors C and a are integrated together and combined into a new vector C' as follows:
C′＝[c1，c2，...[HL]，a1，a2，...a |A| ，[HL]...，c |C| ]，
where HL is a token for indicating the location information and the number of occurrences of an answer in the context.
And 203, inputting the answer content vector and the question vector of each question into the BERT pre-training model, and detecting whether the target answer and each question are matched or not by the BERT pre-training model.
In the embodiment of the invention, the BERT pre-training model can be input by two sentences to finish the sentence pair classification task and detect whether a question is matched with an answer. And when the task of detecting whether the answer is matched with the question is carried out, taking the answer content vector and the question vector of the question as inputs, if the answer content vector is detected to be not matched with the question vector, the target answer is not matched with the question, and if the answer content vector is detected to be matched with the question vector, the target answer is matched with the question. The questions may be generated according to the target answers and the target content, and may specifically be any applicable manner, which is not limited by the embodiment of the present invention. The BERT pre-training model is adopted to train the network by the existing training set A to obtain trained network parameters, then the trained network parameters are stored for later use, when a new task B exists, the same network structure is adopted, the learned parameters of the A can be loaded when the network parameters are initialized, other high-level parameters are randomly initialized, then the training data of the B task are used for training the network, and when the loaded parameters are changed continuously along with the training of the B task, the parameters are better adjusted to be more suitable for the current B task. Therefore, the BERT pre-training model is adopted to facilitate scene migration, and when the new business requirements are faced, a problem generating system applicable to the new business requirements can be quickly built, so that the problem generating efficiency is improved.
For example, with C', the input vector X is designed as follows:
X i ＝([CLS]，C′，[SEP]，q 1 ，...，q i ，[MASK])，
wherein, [ CLS ]]The symbols being used at the beginning of each sequence to represent the classification task, and for two sentences of the input one SEP]The symbols are partitioned. For each generated problem q in each iterative process i The last mark [ MASK ] in the input sequence is taken]As the last hidden state vector h [MASK] E, rh represents the hidden layer vector set while connecting it to a radial layer W HLSQG ∈R h×|V| . Finally, a softmax function is used to calculate the tag probability Pr (w|X i )∈R |v| ) The method is specifically as follows:
where w represents the weight, b represents the offset,representing taking the maximum probability.
If the calculated label probability exceeds the preset threshold, the target answer is matched with the question, and if the calculated label probability does not exceed the preset threshold, the target answer is not matched with the question.
And step 204, obtaining a plurality of candidate questions matched with the target answers.
In the embodiment of the invention, a plurality of candidate questions matched with the target answers are acquired according to the output of the BERT pre-training model.
Step 205, inputting the candidate questions and the target content into a RoBerta pre-training model, and calculating a no-answer score of the answer without the candidate questions in the target content and an answer score of the answer with the candidate questions in the target content.
And step 206, eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions according to the answer-free scores and the answer-free scores to obtain target questions.
In the embodiment of the invention, the problem filtering module based on the RoBerta pre-training model can be adopted for the problem filtering. Inputting the candidate questions and the target content into a trained RoBerta pre-training model, calculating the scores of answers of the candidate questions which do not exist in the target content by the RoBerta pre-training model, marking the scores as no answer scores, and marking the scores of the answers of the candidate questions which exist in the target content as answer scores.
For example, the input question (i.e., candidate question) and paragraph (i.e., target content) are represented as a single compressed sequence of tokens, while a special token [ SEP ] is used]Separating questions from paragraphs. First, a special class mark [ CLS ] is used at the beginning of each sequence]. Denoted by C [ CLS ]]Final hidden representation of the mark, T i Representing the final hidden representation of the i-th input mark. Second, for each question that cannot be answered, [ CLS ] is used]The label indicates the starting and ending index of the answer because it does not have any starting and ending index of the answer. By comparing no-answer span (no answer label) with A no-null answer span score predicts the answer to a question. Wherein, no answer score is calculated as follows: s is S null ＝S.C+E.C。
Where S ε RH is the vector representation of the answer start index and E ε RH is the vector representation of the answer end index. The answer score is calculated as follows: s is S i，j ＝max j＞＝i {S.T i +E.T j }。
Thus, for vector V, the threshold value of V calculated using the validation set, if S null -S i，j > V, the question cannot be answered with the paragraph, otherwise the question can be answered with the paragraph.
In step 207, the target answers with the same stem are grouped to obtain answer question sets consisting of the target answers and the corresponding target questions.
In the embodiment of the invention, after the problem generation and the problem filtering stages, the target answers and the corresponding target problems are obtained. All answers and corresponding questions are grouped, and the specific grouping mode is to divide target answers with the same stem into a group, the target questions corresponding to the target answers are also classified into corresponding groups, and the group consisting of the target answers and the corresponding target questions is recorded as an answer question group. The target answers in the same answer question group have the same stem. For example, the target answer a is "science", the target answer B is "scientist", and the target answer a and the target answer B have the same stem "science", and thus, the target answer a and the target answer B are assigned to the same answer question group.
Step 208, sorting the same answer questions according to the confidence level of the target questions from high to low.
In the embodiment of the invention, for each answer question group, the target questions are ranked according to the confidence level of the target questions generated when the target questions are generated in the answer question group, and the target questions are ranked according to the high-to-low confidence level in the same answer question group, so that when the answer question group is used, the target questions and the target answers with relatively high confidence level appear first, and the target questions and the target answers with relatively low confidence level appear later.
For example, in each group, confidence of the target problem is calculated by normalizing the beam scoreAnd calculating the final inter-problem probability according to the problem with the largest intra-problem probability. Normalizing the confidence of the target problem to obtain the relative confidence of the target problem, wherein p is expressed as +.>Where P represents the maximum probability score set for the answer and P represents the final inter-question probability calculated from the question with the greatest probability within the question.
And step 209, generating a display view of the target answer and the corresponding target questions according to the answer question group.
In the embodiment of the invention, the generated questions and answers are output into a final FAQ (Frequently Asked Questions, common question solutions) set through formatting. The answers and questions may also be displayed, and for each answer question set, a display view of the target answer and the corresponding target question is generated, respectively, i.e., a grouped view of the collection is generated.
According to the embodiment of the invention, the answer content vector representing the target answer and the answer content vector of the target content is combined according to the answer vector of the target answer and the content vector of the target content by receiving the target answer selected from a plurality of candidate answers in the target content or receiving the target answer corresponding to the target position submitted by the target content, the answer content vector and the question vector of each question are input into the BERT pre-training model, whether the target answer and each question are matched is detected by the BERT pre-training model, a plurality of candidate questions matched with the target answer are acquired, the candidate question and the target content are input into the RoBerta pre-training model, the answer-free score of the answer without the candidate question in the target content is calculated, the answer-free score of the candidate question is calculated, rejecting the questions which cannot be answered by adopting the target content in the plurality of candidate questions according to the no-answer score and the answer score to obtain target questions, dividing the target answers with the same word stems into a group to obtain answer question groups consisting of target answers and corresponding target questions, sequencing the target questions in the same answer question groups according to the confidence level of the target questions from high to low, generating a display view of the target answers and the corresponding target questions according to the answer question groups, enabling the answers to be submitted by background personnel, rejecting the questions which cannot be answered by using the answers in the target content in the generated questions, avoiding the problem that the generated questions cannot be answered by using the answers close to the actual service requirements, improving the accuracy of the questions, thereby improving the utilization rate of the generated problems.
Example III
Referring to fig. 3, a block diagram of a problem generating apparatus in a third embodiment of the present invention is shown, which may specifically include:
an answer acquisition module 301, configured to acquire a target answer submitted for a target content;
a question generation module 302, configured to generate a plurality of candidate questions matched with the target answer according to the target answer and the target content;
a score calculating module 303, configured to calculate, according to the plurality of candidate questions and the target content, a no-answer score of an answer in which the candidate question does not exist in the target content, and an answer score of an answer in which the candidate question exists in the target content;
the question rejecting module 304 is configured to reject, according to the answer-free score and the answer-free score, a question that cannot be answered by using the target content from the plurality of candidate questions, so as to obtain a target question;
a providing module 305 is configured to provide the target answer and the corresponding target question.
Optionally, the answer acquisition module includes:
and the receiving sub-module is used for receiving a target answer selected from a plurality of candidate answers in the target content or receiving a target answer corresponding to a target position submitted for the target content.
Optionally, the problem generating module includes:
the vector combination sub-module is used for combining answer content vectors representing the target answers and the target content according to the answer vectors of the target answers and the content vectors of the target content;
the matching detection sub-module is used for inputting the answer content vector and the question vector of each question into the BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
and the question acquisition sub-module is used for acquiring a plurality of candidate questions matched with the target answers.
Optionally, the vector combining submodule includes:
an information acquisition unit for acquiring the position information and the occurrence number of the target answer in the target content;
and a vector composing unit for composing the answer content vector from the answer vector, the content vector, and the position information and the number of occurrences.
Optionally, the score calculating module includes:
the score calculating sub-module is used for inputting the candidate questions and the target content into the RoBerta pre-training model, calculating the unanswered scores of the answers of the candidate questions which are not present in the target content, and calculating the answered scores of the answers of the candidate questions which are present in the target content.
Optionally, the apparatus further comprises:
the grouping module is used for grouping the target answers with the same word stems into a group before the target answers and the corresponding target questions are provided, so as to obtain answer question groups consisting of the target answers and the corresponding target questions;
and the sequencing module is used for sequencing the target questions from high to low according to the confidence degrees of the target questions in the same answer question group.
Optionally, the providing module includes:
and the view generation sub-module is used for generating a display view of the target answer and the corresponding target question according to the answer question group.
According to the embodiment of the invention, the target answers submitted for the target content are obtained, a plurality of candidate questions are generated according to the target answers and the target content, the answer-free scores of the answers of the candidate questions which do not exist in the target content and the answer-free scores of the answers of the candidate questions which exist in the target content are calculated according to the plurality of candidate questions and the target content, the questions which cannot be answered by adopting the target content in the plurality of candidate questions are removed according to the answer-free scores and the answer-free scores, the target questions are obtained, the target answers and the corresponding target questions are provided, so that the answers are submitted by background personnel, the answers are more closely matched with actual demands, the questions which cannot be answered by the answers in the target content in the generated questions are removed, the problems which cannot be answered by the answers close to the actual service demands are avoided, and the accuracy of the questions is improved, and therefore the utilization rate of the generated questions is improved.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
In an embodiment of the disclosure, the problem generating device includes a processor and a memory, where the above modules and sub-modules are stored as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.
The processor includes a kernel, and the kernel fetches the corresponding program unit from the memory. The kernel can set one or more than one of the answers to the target content, generate a plurality of candidate questions according to the target answers and the target content, calculate the answer-free score of the answer without the candidate questions in the target content and the answer-free score of the answer with the candidate questions in the target content according to the target answers and the target content, reject the questions which cannot be answered by the target content in the plurality of candidate questions according to the answer-free score and the answer-free score, obtain the target questions, and provide the target answers and the corresponding target questions, so that the answers are submitted by background personnel, the answers are more relevant to the actual demands, and reject the questions which cannot be answered by the answers in the target content in the generated questions, thereby avoiding the problem that the generated questions cannot be answered by the answers close to the actual service demands, and improving the accuracy of the questions, and further improving the utilization rate of the generated questions.
The memory may include volatile memory, random Access Memory (RAM), and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), among other forms in computer readable media, the memory including at least one memory chip.
An embodiment of the present invention provides a storage medium having stored thereon a program which, when executed by a processor, implements the problem generating method.
The embodiment of the invention provides a processor for running a program, wherein the program runs to execute the problem generating method.
The embodiment of the invention provides an electronic device, which comprises a processor, a memory and a program stored on the memory and capable of running on the processor, wherein the processor realizes the following steps when executing the program:
acquiring a target answer submitted aiming at target content;
generating a plurality of candidate questions matched with the target answers according to the target answers and the target content;
calculating a no-answer score of an answer without the candidate question in the target content and an answer score of an answer with the candidate question in the target content according to the plurality of candidate questions and the target content;
According to the answer-free score and the answer-free score, eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions to obtain target questions;
and providing the target answers and the corresponding target questions.
Optionally, the obtaining the target answer submitted for the target content includes:
and receiving a target answer selected from a plurality of candidate answers in the target content, or receiving a target answer corresponding to a target position submitted for the target content.
Optionally, the generating a plurality of candidate questions matched with the target answer according to the target answer and the target content includes:
combining answer content vectors representing the target answers and the target contents according to the answer vectors of the target answers and the content vectors of the target contents;
inputting the answer content vector and the question vector of each question into the BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
and acquiring a plurality of candidate questions matched with the target answers.
Optionally, the combining the answer vector according to the target answer and the content vector of the target content to form the answer content vector representing the target answer and the target content includes:
Acquiring the position information and the occurrence times of the target answers in the target content;
and forming the answer content vector by the answer vector, the content vector, the position information and the occurrence number.
Optionally, the calculating, according to the plurality of candidate questions and the target content, a no-answer score of an answer in which the candidate questions do not exist in the target content, and the answer score of the answer in which the candidate questions exist in the target content includes:
inputting the candidate questions and target content into a RoBerta pre-training model, and calculating the no-answer score of the answer without the candidate questions in the target content and the answer score of the answer with the candidate questions in the target content.
Optionally, before the providing the target answer and the corresponding target question, the method further includes:
dividing target answers with the same stem into a group to obtain an answer question group consisting of the target answers and corresponding target questions;
and ordering the target questions from high to low in the same answer question group according to the confidence degree of the target questions.
Optionally, the providing the target answer and the corresponding target question includes:
And generating a display view of the target answer and the corresponding target question according to the answer question group.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (8)
1. A problem generating method, comprising:
acquiring a target answer submitted aiming at target content;
generating a plurality of candidate questions matched with the target answers according to the target answers and the target content;
calculating a no-answer score of an answer without the candidate question in the target content and an answer score of an answer with the candidate question in the target content according to the plurality of candidate questions and the target content;
according to the answer-free score and the answer-free score, eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions to obtain target questions;
providing the target answers and corresponding target questions;
the generating a plurality of candidate questions matched with the target answers according to the target answers and the target content comprises:
Combining answer content vectors representing the target answers and the target contents according to the answer vectors of the target answers and the content vectors of the target contents;
inputting the answer content vector and the question vector of each question into a BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
acquiring a plurality of candidate questions matched with the target answers;
the combining the answer vector representing the target answer and the content vector representing the target content according to the answer vector of the target answer and the content vector of the target content comprises:
acquiring the position information and the occurrence times of the target answers in the target content;
and forming the answer content vector by the answer vector, the content vector, the position information and the occurrence number.
2. The method of claim 1, wherein the obtaining the target answer submitted for the target content comprises:
and receiving a target answer selected from a plurality of candidate answers in the target content, or receiving a target answer corresponding to a target position submitted for the target content.
3. The method of claim 1, wherein the calculating, from the plurality of candidate questions and target content, an unanswered score for an answer to the candidate question absent from the target content, and an answered score for an answer to the candidate question present in the target content comprises:
Inputting the candidate questions and target content into a RoBerta pre-training model, and calculating the no-answer score of the answer without the candidate questions in the target content and the answer score of the answer with the candidate questions in the target content.
4. The method of claim 1, wherein prior to said providing the target answer and corresponding target question, the method further comprises:
dividing target answers with the same stem into a group to obtain an answer question group consisting of the target answers and corresponding target questions;
and ordering the target questions from high to low in the same answer question group according to the confidence degree of the target questions.
5. The method of claim 4, wherein the providing the target answer and corresponding target question comprises:
and generating a display view of the target answer and the corresponding target question according to the answer question group.
6. A problem generating apparatus, comprising:
the answer acquisition module is used for acquiring a target answer submitted aiming at the target content;
the question generation module is used for generating a plurality of candidate questions matched with the target answers according to the target answers and the target content;
A score calculating module, configured to calculate, according to the plurality of candidate questions and target content, a no-answer score of an answer in which the candidate question does not exist in the target content, and a answer score of an answer in which the candidate question exists in the target content;
the question eliminating module is used for eliminating the questions which cannot be answered by adopting the target content in the plurality of candidate questions according to the answer-free score and the answer-free score to obtain target questions;
the providing module is used for providing the target answers and the corresponding target questions;
the problem generation module includes:
the vector combination sub-module is used for combining answer content vectors representing the target answers and the target content according to the answer vectors of the target answers and the content vectors of the target content;
the matching detection sub-module is used for inputting the answer content vector and the question vector of each question into a BERT pre-training model, and detecting whether the target answer is matched with each question or not by the BERT pre-training model;
the question acquisition sub-module is used for acquiring a plurality of candidate questions matched with the target answers;
the vector combining submodule includes:
an information acquisition unit for acquiring the position information and the occurrence number of the target answer in the target content;
And a vector composing unit for composing the answer content vector from the answer vector, the content vector, and the position information and the number of occurrences.
7. A storage medium comprising a stored program, wherein the program, when run, controls a device in which the storage medium is located to perform the method of any one of claims 1 to 5.
8. An electronic device, comprising: memory, a processor and executable instructions stored in the memory and executable in the processor, wherein the processor implements the method of any of claims 1-5 when executing the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011106484.XA CN112231456B (en) | 2020-10-15 | 2020-10-15 | Question generation method, device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011106484.XA CN112231456B (en) | 2020-10-15 | 2020-10-15 | Question generation method, device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112231456A CN112231456A (en) | 2021-01-15 |
| CN112231456B true CN112231456B (en) | 2024-02-23 |
Family
ID=74117334
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011106484.XA Active CN112231456B (en) | 2020-10-15 | 2020-10-15 | Question generation method, device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112231456B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920654A (en) * | 2018-06-29 | 2018-11-30 | 泰康保险集团股份有限公司 | A kind of matched method and apparatus of question and answer text semantic |
| CN110688478A (en) * | 2019-09-29 | 2020-01-14 | 腾讯科技（深圳）有限公司 | Answer sorting method, device and storage medium |
| CN111767366A (en) * | 2019-04-01 | 2020-10-13 | 北京百度网讯科技有限公司 | Question and answer resource mining method and device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9721004B2 (en) * | 2014-11-12 | 2017-08-01 | International Business Machines Corporation | Answering questions via a persona-based natural language processing (NLP) system |
-
2020
- 2020-10-15 CN CN202011106484.XA patent/CN112231456B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108920654A (en) * | 2018-06-29 | 2018-11-30 | 泰康保险集团股份有限公司 | A kind of matched method and apparatus of question and answer text semantic |
| CN111767366A (en) * | 2019-04-01 | 2020-10-13 | 北京百度网讯科技有限公司 | Question and answer resource mining method and device, computer equipment and storage medium |
| CN110688478A (en) * | 2019-09-29 | 2020-01-14 | 腾讯科技（深圳）有限公司 | Answer sorting method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112231456A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111291570B (en) | Method and device for realizing element identification in judicial documents | |
| CN109684476B (en) | Text classification method, text classification device and terminal equipment | |
| CN112686022A (en) | Method and device for detecting illegal corpus, computer equipment and storage medium | |
| WO2022154897A1 (en) | Classifier assistance using domain-trained embedding | |
| CN111124487A (en) | Code clone detection method and device and electronic equipment | |
| CN110610003B (en) | Method and system for assisting text annotation | |
| CN113722483A (en) | Topic classification method, device, equipment and storage medium | |
| CN114625858A (en) | Intelligent government affair question-answer replying method and device based on neural network | |
| CN112287100A (en) | Text recognition method, spelling error correction method and voice recognition method | |
| CN112464927B (en) | Information extraction method, device and system | |
| CN114218945A (en) | Entity identification method, device, server and storage medium | |
| CN113222022A (en) | Webpage classification identification method and device | |
| CN115374259A (en) | Question and answer data mining method and device and electronic equipment | |
| CN113934834A (en) | Question matching method, device, equipment and storage medium | |
| JP2013250926A (en) | Question answering device, method and program | |
| CN116340511B (en) | Public opinion analysis method combining deep learning and language logic reasoning | |
| Vu et al. | Revising FUNSD dataset for key-value detection in document images | |
| CN109902162B (en) | Text similarity identification method based on digital fingerprints, storage medium and device | |
| CN112231456B (en) | Question generation method, device, storage medium and electronic equipment | |
| CN111209724A (en) | Text verification method and device, storage medium and processor | |
| CN107729509A (en) | The chapter similarity decision method represented based on recessive higher-dimension distributed nature | |
| US9443139B1 (en) | Methods and apparatus for identifying labels and/or information associated with a label and/or using identified information | |
| JP5824429B2 (en) | Spam account score calculation apparatus, spam account score calculation method, and program | |
| CN105808522A (en) | Method and apparatus for semantic association | |
| CN111242060A (en) | Method and system for extracting key information of document image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |