CN115713065A - Method for generating question, electronic equipment and computer readable storage medium - Google Patents
Method for generating question, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115713065A CN115713065A CN202211393614.1A CN202211393614A CN115713065A CN 115713065 A CN115713065 A CN 115713065A CN 202211393614 A CN202211393614 A CN 202211393614A CN 115713065 A CN115713065 A CN 115713065A
- Authority
- CN
- China
- Prior art keywords
- answer
- text
- question
- generated
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a method for generating a problem, an electronic device and a computer readable storage medium: providing a text of a question to be generated and providing an answer of the question to be generated; inputting the relevant information of the answer into a first neural network to extract features, and obtaining the relevant features of the answer; inputting the relevant information of the answer into a second neural network to calculate the relevance, and obtaining the relevant characteristic information of the answer and the question to be generated; inputting the relevant information of the text into a third neural network to extract features, and obtaining the relevant feature information of the text and the problem to be generated; inputting relevant features of the answers of the questions to be generated, relevant feature information of the answers and the questions to be generated, and relevant feature information of the texts and the questions to be generated into a neural network model for generating the questions to calculate, and obtaining words and sentences in the texts as word and sentence probability values in the questions to be generated; and selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability value of the words and sentences in the text as the problem to be generated. The method and the device accurately generate the question of the corresponding answer.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method for generating a question, an electronic device, and a computer-readable storage medium.
Background
The question-answering system based on the knowledge base can give corresponding answers aiming at questions posed by clients. The method comprises the steps that a corresponding relation between questions and answers is set in a knowledge base of a question-answering system based on the knowledge base, when the question-answering system receives the questions put forward by a client, the questions are matched with the set questions, answers corresponding to the successfully matched questions are obtained from the knowledge base and are provided for the client.
In the knowledge base of the question-answering system based on the knowledge base, when the corresponding relation between the question and the answer is set, how to extract the question from the text or extract the answer from the text is to correspond the question and the answer, and the setting of the knowledge base is a problem to be solved urgently. Taking the example of extracting answers from text, the answers may be extracted from text in various ways. One way is as follows: the answer is obtained from the text in a manual mode based on the set template, so that a large amount of manpower is required, the expansibility is not strong, the generalization is not easy, the efficiency is low, and the standard consistency is poor. The other mode is as follows: the method comprises the steps of inputting each sentence in a text and text features related to the sentence into a neural network model for processing by adopting the neural network model obtained through training, obtaining a similarity value of each sentence, and taking the sentence with the highest similarity value in the text as an answer, wherein the text features related to the sentence are position features of the sentence in the text, lexical features of the sentence in the text, or/and lexical features of the sentence in the text. However, this method obtains answers on the premise that the default answer is strongly related to the position feature, lexical feature and lexical feature of the answer appearing in the text, but not in the real scene, so the obtained answer is not accurate.
The above method focuses on how to obtain answers from texts, and the answers are applied to a knowledge base in the question-answering system. Similarly, questions in the knowledge base of the question-answering system can also be obtained in the above manner. However, when the above method is used to obtain the questions in the knowledge base of the question-answering system, the method is not accurate, and the characteristic information of the questions in the knowledge base, such as expression mode and semantics, directly affects the matching success rate of matching with the questions provided by the customer, thereby affecting the user experience of the customer using the question system based on the knowledge base.
Disclosure of Invention
In view of this, embodiments of the present application provide a method, an electronic device, and a computer-readable storage medium for generating questions, which can accurately generate questions corresponding to answers in a question-answering system based on a knowledge base, so that a matching rate is improved when subsequently matching questions posed by a client.
In one embodiment of the embodiments of the present application, a method for generating a question is provided, the method including:
providing a text of a question to be generated and providing an answer of the question to be generated;
inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer;
inputting the relevant information of the answer into a second neural network for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated;
inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated;
inputting the answer correlation characteristics of the question to be generated, the correlation characteristic information of the answer and the question to be generated and the correlation characteristic information of the text and the question to be generated into a neural network model for generating the question to calculate the probability of words and sentences in the text as words and sentences in the question to be generated so as to obtain the probability value of words and sentences in the text as words and sentences in the question to be generated;
and selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability value of the words and sentences in the text as the problem to be generated.
In the above method, the first neural network is implemented by using a self-attention mechanism, and the relevant features of the answer include: each word feature, keyword feature, location feature or/and answer semantic feature of the answer.
In the above method, the second neural network is implemented using a supervised contrast learning neural network.
In the above method, the inputting the relevant information of the answer into the second neural network for correlation calculation includes:
the supervised contrast learning neural network obtains semantic features of the answers and segment features of the answers from the relevant information of the answers, and cosine similarity calculation is carried out on the semantic features of the answers and the segment features of the answers, wherein the segment features of the answers comprise previous sentences of the answers in the text, current sentences of the answers in the text and next sentences of the answers in the text;
taking the cosine similarity value obtained by calculation as the similarity value between the answer and the question to be generated;
and the similarity value of the answer and the question to be generated is the correlation characteristic information of the answer and the question to be generated.
In the method, the third neural network is implemented by using a relational memory neural network R-Men.
In the above method, the inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated includes:
inputting position information and feature information of triple features in the related information of the text into an R-MeN for input processing, then extracting the features by the R-MeN by adopting a set self-attention mechanism network, and then decoding and calculating the extracted features by the R-MeN by adopting a set Convolutional Neural Network (CNN) to obtain triple validity scores of the related information of the text, wherein the triple features of the related information of the text comprise the text features, relationship features between the text features and answer features, and the answer features;
and taking the triple validity score value of the relevant information of the text as the relevant characteristic information of the text and the question to be generated.
In the method, the neural network model for generating the problem is realized by adopting a gate control training unit GRU architecture or a long-short term memory artificial neural network LSTM.
The method further comprises the following steps:
and setting the formed question to be generated in a knowledge base in a question-answering system so that the knowledge base in the question-answering system sets the corresponding relation between the formed question to be generated and the answer.
Another embodiment of the present application provides an electronic device, including:
a processor;
a memory storing a program configured to implement any of the above methods of generating a question when executed by the processor.
In yet another embodiment of the present application, a non-transitory computer readable storage medium is provided that stores instructions that, when executed by a processor, cause the processor to perform any of the above-described methods of generating a question.
As seen from the above, the scheme adopted in the embodiment of the present application is as follows: providing a text of a question to be generated and providing an answer of the question to be generated; inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer; inputting the relevant information of the answer into a second neural network for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated; inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated; inputting the answer correlation characteristics of the question to be generated, the correlation characteristic information of the answer and the question to be generated and the correlation characteristic information of the text and the question to be generated into a neural network model for generating the question to calculate the probability of words and sentences in the text as words and sentences in the question to be generated so as to obtain the probability value of words and sentences in the text as words and sentences in the question to be generated; and selecting a set number of words and sentences in the text from the high and low sequence based on the probability value of the words and sentences in the text as the problem to be generated to form the problem to be generated. Therefore, when the questions are generated, the three types of texts relevant to the generated questions and the characteristic information related to the answers can be obtained, and accurate calculation is carried out according to the characteristic information, so that the questions can be accurately generated, and the generated questions can be applied to a knowledge base in a question-answering system. Therefore, the question corresponding to the answer in the question-answering system based on the knowledge base is accurately generated, and the matching rate is improved when the question proposed by the customer is subsequently matched.
Drawings
Fig. 1 is a schematic diagram of a neural network model architecture for setting a correspondence between questions and answers in some embodiments;
FIG. 2 is a flowchart of a method for generating questions provided by an embodiment of the present application;
fig. 3 is a schematic structural diagram of a third neural network processing information related to the text according to an embodiment of the present application;
FIG. 4 is a system architecture diagram for generating questions for implementation provided herein;
FIG. 5 is a schematic structural diagram of an apparatus for generating a question according to an embodiment of the present application;
fig. 6 is a schematic diagram of an electronic device according to another embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The technical solution of the present application will be described in detail with specific examples. Several of the following embodiments may be combined with each other and some details of the same or similar concepts or processes may not be repeated in some embodiments.
At present, when answers or questions are set in a knowledge base of a question-answering system based on the knowledge base, the answers or the questions can be realized by adopting a neural network obtained by training. The setting of the answer is described by taking the example of extracting the answer from the text. Fig. 1 is a schematic diagram of a neural network model architecture adopted by a knowledge base in a question-answering system to set answers in some technical solutions. As shown in the figure, the Answer is included in the text, the characteristics of each word (word) in the text, the Position characteristics (Answer Position Features) of the word in the text, the semantic characteristics (Lexical characteristics) of the word in the text (Lexical characteristics including the Lexical characteristics of the word in the text and the Lexical characteristics of the word in the text) and the like of the word in the text are input into the trained neural network model for processing, the similarity value of each sentence is obtained, the sentence with the highest similarity value in the text is used as the Answer, and the Answer is set in the knowledge base. The neural network model obtained by training is realized by adopting an attention mechanism neural network.
The precondition for setting up the answers of the knowledge base by using the process described in fig. 1 is that the default answers are strongly correlated with the position features, lexical features, and lexical features of the answers appearing in the text, but not in the real scene, so the obtained answers are not accurate. In order to solve the problem, a scheme of processing by adopting a trained neural network model after extracting the relational features is provided. In the scheme, for each sentence in a text, the segment characteristics of the sentence in the text and the relationship characteristics between other sentences in the text and the sentence are extracted to obtain the relationship characteristics of the sentence, then the sentence and the sentence are input into a trained neural network model to be processed, the similarity values of the sentence and other sentences in the text are output, and the sentence with the highest similarity value in the text is used as an answer and is arranged in a knowledge base. The method can improve the accuracy of the set answer to a certain extent, but the extraction rule is set manually, so the types of the extracted relational features are limited, and an open-source relational extraction model OpenIE is adopted when the relational features are extracted, so the problem of extraction error accumulation exists, and the finally set answer is not accurate.
Of course, the questions can be obtained from the text by adopting the above mode and are arranged in the knowledge base of the question-answering system. In this case, there is also a problem that the set problem is inaccurate. When the knowledge base of the question-answering system is constructed, the coverage of the question-answering system is improved, and an important link in a question-answering task of a user side is met, and the problem that answers or questions are inaccurate when the knowledge base is set is solved. For example, when the question-answering system is applied to the field of house property sales, the customer's question is basically a question caused by the fact that the knowledge of some house property policy is unknown, such as "can you buy house locally in a different place? "how long a public accumulation loan period takes", etc., these questions are actually in the national policy and documentation, and the corresponding answers to these questions are also embodied in the text. In this case, however, it is impossible to determine what expression form of question the customer has asked, and the existing data exists in the form of a "standard question-text-answer" structure. Therefore, the questions meeting the questions asked by the clients need to be generated through the existing data in the structural form of "text-answer" and are set in the knowledge base of the question-answering system. Usually, the generation is performed manually, or by referring to the process shown in fig. 1 or the improved process shown in fig. 1, but in this case, the generated problem is inaccurate, the questioning requirement of the customer cannot be met, and the user experience is not good.
In order to solve the above problem, the embodiment of the present application adopts a scheme that: providing a text of a question to be generated and providing an answer of the question to be generated; inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer; inputting the relevant information of the answer into a second neural network for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated; inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated; inputting the relevant characteristics of the answer, the relevant characteristic information of the answer and the question to be generated and the relevant characteristic information of the text and the question to be generated into a neural network model for generating the question to calculate the probability of the words and the sentences in the text as the words and the sentences in the question to be generated so as to obtain the probability value of the words and the sentences in the text as the words and the sentences in the question to be generated; and selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability values of the words and sentences in the text as the problem to be generated.
Therefore, when the questions are generated, the three types of texts relevant to the generated questions and the characteristic information related to the answers can be obtained, accurate calculation is carried out according to the three types of texts relevant to the generated questions, and the generated questions can be accurately generated and applied to a knowledge base in a question-answering system.
Therefore, the question corresponding to the answer in the question-answering system based on the knowledge base is accurately generated, and the accuracy is improved when the question provided by the client is subsequently matched.
The method and the device combine the plurality of constructed neural networks with a neural network model for generating the questions, and process information including texts and answers to generate the questions set by a knowledge base of a question-answering system. Based on the key context information in the text, a third neural network, namely a relation memory network (R-Men) is adopted for processing, so that the correlation characteristic information of the text and the problem to be generated can be obtained, and the accuracy of the subsequent problem generation is improved; based on the answer information and the text information, a second neural network, namely a supervision comparison learning neural network is adopted to calculate the similarity (to similarity), so that the correlation characteristic information of the answers and the questions to be generated can be described, and the accuracy of the subsequently generated questions is improved based on the answer information; the three types of obtained characteristic information are integrated by adopting a neural network model with a gated cycle concept for generating problems and probability value calculation is carried out, so that the obtained three-dimensional characteristic information is effectively utilized when the problems are generated, the accuracy of the generated problems is improved, and the quality of the generated problems is better.
Fig. 2 is a flowchart of a method for generating a question according to an embodiment of the present application, which includes the specific steps of:
step 202, inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer;
step 204, inputting the relevant information of the text into a third neural network for feature extraction to obtain relevant feature information of the text and the problem to be generated;
and step 206, selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability values of the words and sentences in the text as the problem to be generated.
In the above method, the set number is set as needed, and is not limited herein.
In the process, when the set number is 1, words and sentences with the highest probability value are extracted from the text, and a problem to be generated is formed.
In this embodiment of the present application, the relevant feature information of the answer is a feature obtained by encoding using a first neural network, the relevant feature information of the answer and the to-be-generated question is a feature obtained by encoding using a second neural network, and the relevant feature information of the text and the to-be-generated question is a feature obtained by encoding using a third neural network. In the three neural networks, a self-attention mechanism (self-attention) is adopted, so that corresponding characteristic information can be extracted respectively.
In the embodiment of the present application, the first neural network, the second neural network, the third neural network, and the neural network model for generating the problem are obtained by training using a training data source. Here, the training data source includes text, answers included in the text, and questions included in the text, wherein the questions included in the text are used for subsequently verifying whether the generated questions are accurately used.
In this embodiment of the present application, the relevant information of the text is obtained by preprocessing the provided text to be generated with a question, where the preprocessing includes two steps: the first step, cleaning stop words and punctuation marks in the text; and a second step of intercepting the length of the text, wherein the length of the text is long and is usually about 380 characters, and the sentence with the set number of characters is intercepted forwards and backwards by taking the sentence including the answer as a center so as to ensure that the answer is in the intercepted sentence and has context information.
Similarly, in the embodiment of the present application, the answer may be further preprocessed to obtain information related to the answer, and when the answer is preprocessed, stop words and punctuation marks in the answer may be cleaned.
Take a specific example for illustration. The text of the question to be generated includes: the merchant loan currently deposits money for two to three months, and specifically, whether the bank policies are tightened or not is determined. The loan of the pure public accumulation fund is released more quickly, about two months. The combination credits back more slowly, perhaps three to four months. The answer to the question to be generated is: the pure public accumulation fund loan is released about two months. The problems to be generated are: time for putting money on the pure public accumulation fund?
In this case, the text needs to be preprocessed to obtain the relevant information of the text, and the answer needs to be preprocessed to obtain the relevant information of the answer.
In an embodiment of the present application, the first neural network is implemented by using an attention mechanism, and the relevant features of the answer include: each word feature, keyword feature, location feature or/and answer semantic feature of the answer. Here, the first neural network encodes the relevant information of the answer, and obtains the basic features of the answer to be processed as the input of the neural network model for generating the question.
In the embodiment of the present application, the second neural network is implemented by using a Sentence-embedded supervised contrast Learning (simcse) network. The second neural network can encode the relevant information of the answer by adopting the existing simcse encoding mode, and in order to be more applicable to encoding, a fine tuning (finetune) technology is adopted to fine tune the encoding mode of the existing simcse network for reuse.
Specifically, the inputting of the relevant information of the answer into the second neural network for correlation calculation includes:
the simcse network obtains the semantic features of the answer and the segment features of the answer from the relevant information of the answer, and performs cosine similarity calculation on the semantic features of the answer and the segment features of the answer;
taking the cosine similarity value obtained by calculation as the similarity value between the answer and the question to be generated;
and the similarity value of the answer and the question to be generated is the correlation characteristic information of the answer and the question to be generated.
Here, the semantic features of the answer and the segment features of the answer are extracted by the simcse network by using a self-attention mechanism, and the segment features of the answer include relevant features of a question to be generated. Specifically, the segment characteristics of the answer comprise a previous sentence of the answer in the text, a current sentence of the answer in the text and a next sentence of the answer in the text, and the segment information of the answer is extracted through the attention mechanism of the simcse network.
Here, the cosine similarity calculation between the semantic features of the answer and the segment features of the answer adopts the following formula:
wherein, A is the segment characteristic of the answer, namely the relevant characteristic of the question to be generated, Q is the semantic characteristic of the answer, and the two characteristics are coded and mapped in the simcse network.
In the embodiment of the present application, the third neural network is implemented by using a relational memory neural network (R-MeN). Specifically, inputting the relevant information of the text into a third neural network for feature extraction, and obtaining the relevant feature information of the text and the problem to be generated includes: inputting position information and feature information of triple features (s.r.o) in the related information of the text into an R-MeN for input processing, wherein the R-MeN adopts a set self-attention mechanism network to perform feature extraction, and the R-MeN further adopts a set Convolutional Neural Network (CNN) to perform decoding calculation on the extracted features to calculate a triple validity score value of the related information of the text, wherein the triple features of the related information of the text comprise the text features, relationship features between the text features and answer features, and the answer features; and taking the triple effectiveness score value of the text as the correlation characteristic information of the text and the question to be generated. The arranged self-attention mechanism network is a multilayer feedforward neural network.
Here, fig. 3 is a schematic diagram of an architecture of the third neural network provided in the embodiment of the present application for processing the relevant information of the text. As shown in the figure, three gray circles represent feature information (embedding) of a triple feature in the related information of the text, three white circles identify position information (positional encoding) of a triple feature in the related information of the text, and the position information and the feature information of a triple feature (s, R, o) in the related information of the text are input into the R-MeN and input processing is performed by using the following formula:
x 1 ＝W(v s +p 1 )+b (2)
x 2 ＝W(v r +p 2 )+b
x 3 ＝W(v o +p 3 )+b
V s ，V r ，V o feature vector representation, p, referring to triplet features (s, r, o) i Refers to a position vector. Obtained x i Input vector, x, referring to R-MeX network 1 、x 2 And x 3 And respectively representing the calculated triple features (s, r, o) as input parameters of the self-attention mechanism network.
Wherein W represents a weight, and b represents a set bias coefficient，v s ，v r ，v o Respectively representing feature vectors of the triple features (s, r, o), p representing position information of each word in the relevant information of the text, subscript s being a text feature, subscript r being a relation information feature between the feature and an answer feature, and subscript o being an answer feature. Finally obtain x 1 、x 2 And x 3 As input parameters for the self-attention mechanism network in R-MeN.
As shown in fig. 3, x will be obtained 1 、x 2 And x 3 After multilayer feedforward (MLF) processing of the self-attention mechanism is performed on the text features, the relationship features between the text features and the answer features, feature extraction results shown by three striped circles in fig. 3 are obtained, and the feature extraction results are sent to the CNN for decoding processing, so that triple validity score values of the text are obtained. In the CNN, the triple effectiveness score value of the text is obtained by using the following formula:
f(s，r，o)＝max(ReLU([y 1 ，y 2 ，y 3 ]*Ω)) T w (3)
wherein, y 1 、y 2 And y 3 Is X in the formula (2) 1 、X 2 And X 3 After the attention mechanism processing in the third neural network, the obtained vector representation is based on y 1 、y 2 And y 3 Calculating a triple validity score value for the text.
In the embodiment of the present application, the neural network model for generating the problem is implemented by using a gated training unit (GRU) architecture or a long-short term memory artificial neural network (LSTM), where w of the above formula represents a weight vector, and Ω represents a set filter set, and belongs to R m*3 Denotes a convolution operation. The three types of feature information obtained above are effectively combined and effectively utilized through the neural network model for generating the question (realized by the RELU function of formula (3)), so that the neural network model for generating the question performs probability calculation of words and phrases in the text as words and phrases in the question to be generated, and obtains the words and phrases in the text as the question to be generatedAnd the word and sentence probability values in the question are used for extracting words and sentences with the highest probability values from the text subsequently to form a question to be generated.
Here, the neural network model for generating the question is implemented by using the following formula, which gives the probability that the nth word in the text is the nth word of the question based on the three types of feature information obtained as described above, and is calculated by using the following formula (4):
wherein p is v (ω) represents the probability, p, that the relevant feature of the answer generates a question ω s (ω) represents a probability, p, of the question ω of the answer matching the relevance feature information of the question to be generated M (ω) represents a probability that the text and the correlation characteristic information of the question to be generated generate a question ω,
respectively, representing the resulting vectors calculated by the gating means in the network architecture (fig. 4). The embodiment of the application aims to maximize the probability that the generated words and sentences are connected into a sentence problem.
Fig. 4 shows a system architecture diagram of the method shown in fig. 2, and fig. 4 is a system architecture diagram for realizing a problem generation provided in the present application. The whole process is shown in figure 4:
the method comprises a first step of inputting relevant information of an answer into a first neural network with a self-attention mechanism for feature extraction, and obtaining relevant features of the answer, wherein the relevant features of the answer comprise: each word feature, keyword feature, location feature or/and answer semantic feature of the answer, and the relevant features of the answer are shown in fig. 4;
inputting the relevant information of the answer into a second neural network with a self-attention mechanism for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated; the relevance characteristics of the answers to the questions are shown in fig. 4;
inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated; the relevance features of the text to the question are shown in FIG. 4;
and fourthly, inputting the relevant characteristics of the answer, the relevant characteristic information of the answer and the question to be generated and the relevant characteristic information of the text and the question to be generated into a neural network model for generating the question, wherein the model adopts a gating mode to carry out probability calculation on each word in the text as a word in the question to be generated so as to obtain a word probability value of each word in the text as a word in the question to be generated.
In the fourth step, specifically, the neural network model for generating the question is implemented by using the above formula (4), and in fig. 4, after the probability of the question ω (calculated by using self-attention of the relevant feature of the accessed answer and shown as a gray line box representing the probability of the question ω) is generated from the relevant feature of the answer, the method proceeds
(iii) calculation of (c) (represented in fig. 4 by the gating approach to control calculation); after the probability of the question ω (calculated by self-attention of the correlation features of the access answer and the question and shown as the gray line frame of the probability of the question ω in fig. 4) representing the matching of the answer and the correlation feature information of the question to be generated is obtained, the process is performed
(in fig. 4, the control calculation is represented in a gating manner), after the probability of the problem ω is generated from the text and the correlation characteristic information of the problem to be generated (in fig. 4, self-attention of the correlation characteristic of the text and the problem is calculated, and the illustrated gray line box is represented by the probability of the problem ω), the calculation is performed
(represented in FIG. 4 by the gating mode of control calculation); and adding the calculated result vectors (shown by histograms with different gray scales in fig. 4) to obtain a word probability value of each word in the text as a question to be generated.
The embodiment of the invention can generate accurate problems by adopting the process and improve the quality of the generated problems. The embodiment of the application can apply the generated questions to the knowledge base of the question-answering system, so that the questions can be automatically matched and used after the questions are provided by the client. Specifically, the method further comprises: and setting the formed question to be generated in a knowledge base in a question-answering system so that the knowledge base in the question-answering system sets the corresponding relation between the formed question to be generated and the answer.
After the questions are generated and arranged in the knowledge base of the question answering system, the generated questions can be evaluated. In the specific evaluation, two modes can be adopted, as described in detail below.
The first mode is as follows: automatic abstract evaluation mode (ROUGE-L)
L in ROUGE-L refers to the Longest Common Subsequence (LCS), and the ROUGE-L is calculated using the longest common subsequence of machine translation C and reference translation S, and is calculated as follows:
wherein R in the formula LCS Represents the recall rate of the generated question, and P LCS Accuracy of the representation of the problem, F LCS And C and S in the calculation respectively represent a text generated by a machine and a reference text. In the formula, β will be set to infinity, and thus, F LCS Is actually R LCS . After ROUGE-L evaluation, F is obtained LCS Reaching 0.73%.
The second mode is as follows: manual evaluation mode
In order to ensure the online effect, a manual leveling method is added after the ROUGE-L is adopted for evaluation. In the generated problems, 500 pieces of data are randomly selected, after labeling is carried out, the labeled problems are evaluated, and the problem accuracy of matched clients is used as an evaluation index. After manual evaluation, the matching accuracy of the labeled problem reaches 66.2%.
Therefore, it can be seen from the result obtained by evaluation that the problem is generated by adopting the embodiment of the application, the accuracy and the quality of the generated problem are improved, and the problem generation efficiency is also improved.
Fig. 5 is a schematic structural diagram of a device for generating a question according to an embodiment of the present application, where the device includes: an acquisition unit, a processing unit and a problem generation unit, wherein,
the device comprises an acquisition unit, a processing unit and a display unit, wherein the acquisition unit is used for providing a text of a question to be generated and providing an answer of the question to be generated;
the processing unit is used for inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer; inputting the relevant information of the answer into a second neural network for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated; inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated;
a question generating unit, configured to input the answer related feature of the question to be generated, the relevance feature information of the answer and the question to be generated, and the relevance feature information of the text and the question to be generated into a neural network model for generating a question, perform probability calculation on words and sentences in the text as words and sentences in the question to be generated, and obtain a word and sentence probability value in the text as words and sentences in the question to be generated; and selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability value of the words and sentences in the text as the problem to be generated.
In another embodiment of the present application, there is also provided an electronic device, including: a processor; a memory storing a program configured to implement the method of generating a question as described above when executed by the processor.
In another embodiment of the present application, a non-transitory computer readable storage medium is provided that stores instructions that, when executed by a processor, cause the processor to perform the matched text method of the preceding embodiments. Fig. 6 is a schematic diagram of an electronic device according to another embodiment of the present application. As shown in fig. 6, another embodiment of the present application further provides an electronic device, which may include a processor 601, where the processor 601 is configured to execute the steps of one of the above-mentioned methods for generating a question. As can also be seen from fig. 6, the electronic device provided by the above embodiment further comprises a non-transitory computer readable storage medium 602, on which the non-transitory computer readable storage medium 702 stores a computer program, which when executed by the processor 601 performs the steps of the above-described method of generating a problem.
In particular, the non-transitory computer readable storage medium 602 can be a general purpose storage medium such as a removable disk, a hard disk, a FLASH, a Read Only Memory (ROM), an erasable programmable read only memory (EPROM or FLASH memory), or a portable compact disc read only memory (CD-ROM), etc., and the computer program on the non-transitory computer readable storage medium 602, when executed by the processor 601, can cause the processor 601 to perform the steps of one of the above-described methods of generating a problem.
In practical applications, the non-transitory computer readable storage medium 602 may be included in the device/apparatus/system described in the above embodiments, or may exist separately without being assembled into the device/apparatus/system. The computer readable storage medium carries one or more programs which, when executed, perform the steps of a method of generating a problem as described above.
Yet another embodiment of the present application also provides a computer program product comprising a computer program or instructions which, when executed by a processor, performs the steps of a method of generating a problem as described above.
The flowchart and block diagrams in the figures of the present application illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments disclosed herein. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It will be appreciated by those skilled in the art that various combinations and/or combinations of features recited in the various embodiments of the disclosure and/or in the claims may be made even if such combinations or combinations are not explicitly recited in the present application. In particular, the features recited in the various embodiments and/or claims of the present application may be combined and/or coupled in various ways, all of which fall within the scope of the present disclosure, without departing from the spirit and teachings of the present application.
The principles and embodiments of the present application are explained herein using specific examples, which are provided only for the purpose of facilitating understanding of the method and the core idea of the present application and are not intended to limit the present application. It will be appreciated by those skilled in the art that changes may be made in this embodiment and its applications without departing from the spirit and scope of the invention, and that the invention includes all such modifications, equivalents, and improvements as fall within the true spirit and scope of the invention.
Claims (10)
1. A method of generating a question, the method comprising:
providing a text of a question to be generated and providing an answer of the question to be generated;
inputting the relevant information of the answer into a first neural network for feature extraction to obtain the relevant features of the answer;
inputting the relevant information of the answer into a second neural network for correlation calculation to obtain the relevant characteristic information of the answer and the question to be generated;
inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the problem to be generated;
inputting the answer correlation characteristics of the question to be generated, the correlation characteristic information of the answer and the question to be generated and the correlation characteristic information of the text and the question to be generated into a neural network model for generating the question to calculate the probability that words and sentences in the text are used as words and sentences in the question to be generated so as to obtain the probability value that words and sentences in the text are used as words and sentences in the question to be generated;
and selecting a set number of words and sentences in the text to form the problem to be generated based on the high-low sequence of the probability value of the words and sentences in the text as the problem to be generated.
2. The method of claim 1, wherein the first neural network is implemented using a self-attention mechanism, and the relevant features of the answer include: each word feature, keyword feature, location feature or/and answer semantic feature of the answer.
3. The method of claim 1, in which the second neural network is implemented using a supervised contrast learning neural network.
4. The method of claim 3, wherein said inputting the relevant information of the answer into a second neural network for correlation computation comprises:
the supervised contrast learning neural network acquires semantic features of the answer and segment features of the answer from the relevant information of the answer, and performs cosine similarity calculation on the semantic features of the answer and the segment features of the answer, wherein the segment features of the answer comprise a previous sentence of the answer in the text, a current sentence of the answer in the text and a next sentence of the answer in the text;
taking the cosine similarity value obtained by calculation as the similarity value between the answer and the question to be generated;
and the similarity value of the answer and the question to be generated is the correlation characteristic information of the answer and the question to be generated.
5. The method of claim 1, in which the third neural network is implemented using a relational memory neural network, R-MeN.
6. The method of claim 5, wherein inputting the relevant information of the text into a third neural network for feature extraction to obtain the relevant feature information of the text and the question to be generated comprises:
inputting position information and feature information of triple features in the relevant information of the text into an R-MeN for input processing, then extracting the features by the R-MeN by adopting a set self-attention mechanism network, and then decoding and calculating the extracted features by the R-MeN by adopting a set Convolutional Neural Network (CNN) to obtain triple effectiveness scores of the relevant information of the text, wherein the triple features of the relevant information of the text comprise the text features, relationship features between the text features and answer features, and the answer features;
and taking the triple effectiveness score value of the relevant information of the text as the relevant characteristic information of the text and the question to be generated.
7. The method of claim 1, in which the neural network model of the problem is implemented using a gated training unit GRU architecture or a long-short term memory artificial neural network LSTM.
8. The method of claim 1, wherein the method further comprises:
and setting the formed questions to be generated in a knowledge base in a question answering system so that the knowledge base in the question answering system sets the corresponding relation between the formed questions to be generated and the answers.
9. An electronic device, comprising:
a processor;
a memory storing a program configured to implement the method of generating a question as claimed in any one of claims 1 to 7 when executed by the processor.
10. A non-transitory computer readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of generating a question of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211393614.1A CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211393614.1A CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115713065A true CN115713065A (en) | 2023-02-24 |
| CN115713065B CN115713065B (en) | 2023-09-15 |
Family
ID=85232602
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211393614.1A Active CN115713065B (en) | 2022-11-08 | 2022-11-08 | Method for generating problem, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115713065B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766423A (en) * | 2018-12-29 | 2019-05-17 | 上海智臻智能网络科技股份有限公司 | Answering method and device neural network based, storage medium, terminal |
| CN110188176A (en) * | 2019-04-30 | 2019-08-30 | 深圳大学 | Deep learning neural network and training, prediction technique, system, equipment, medium |
| CN111324721A (en) * | 2020-03-16 | 2020-06-23 | 云南电网有限责任公司信息中心 | Method for constructing intelligent question-answering knowledge base |
| CN112214593A (en) * | 2020-11-05 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Question and answer processing method and device, electronic equipment and storage medium |
| US20210256018A1 (en) * | 2018-04-23 | 2021-08-19 | Nippon Telegraph And Telephone Corporation | Question responding apparatus, question responding method and program |
| WO2021164284A1 (en) * | 2020-02-19 | 2021-08-26 | 平安科技（深圳）有限公司 | Method, apparatus and device for generating reading comprehension question, and storage medium |
| WO2021184311A1 (en) * | 2020-03-19 | 2021-09-23 | 中山大学 | Method and apparatus for automatically generating inference questions and answers |
| WO2021217935A1 (en) * | 2020-04-29 | 2021-11-04 | 深圳壹账通智能科技有限公司 | Method for training question generation model, question generation method, and related device |
| US20210406467A1 (en) * | 2020-06-24 | 2021-12-30 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for generating triple sample, electronic device and computer storage medium |
-
2022
- 2022-11-08 CN CN202211393614.1A patent/CN115713065B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210256018A1 (en) * | 2018-04-23 | 2021-08-19 | Nippon Telegraph And Telephone Corporation | Question responding apparatus, question responding method and program |
| CN109766423A (en) * | 2018-12-29 | 2019-05-17 | 上海智臻智能网络科技股份有限公司 | Answering method and device neural network based, storage medium, terminal |
| CN110188176A (en) * | 2019-04-30 | 2019-08-30 | 深圳大学 | Deep learning neural network and training, prediction technique, system, equipment, medium |
| WO2021164284A1 (en) * | 2020-02-19 | 2021-08-26 | 平安科技（深圳）有限公司 | Method, apparatus and device for generating reading comprehension question, and storage medium |
| CN111324721A (en) * | 2020-03-16 | 2020-06-23 | 云南电网有限责任公司信息中心 | Method for constructing intelligent question-answering knowledge base |
| WO2021184311A1 (en) * | 2020-03-19 | 2021-09-23 | 中山大学 | Method and apparatus for automatically generating inference questions and answers |
| WO2021217935A1 (en) * | 2020-04-29 | 2021-11-04 | 深圳壹账通智能科技有限公司 | Method for training question generation model, question generation method, and related device |
| US20210406467A1 (en) * | 2020-06-24 | 2021-12-30 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for generating triple sample, electronic device and computer storage medium |
| CN112214593A (en) * | 2020-11-05 | 2021-01-12 | 腾讯科技（深圳）有限公司 | Question and answer processing method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 曾碧卿，裴枫华，徐马一，丁美荣: "基于双注意力的段落级问题生成研究", 中文信息学报, vol. 36, no. 8 * |
| 肖康，周夏冰，王中卿，段湘煜，周国栋，张民: "基基于产品建模的评论问题生成研究", 计算机科学, vol. 49, no. 2 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115713065B (en) | 2023-09-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11120801B2 (en) | Generating dialogue responses utilizing an independent context-dependent additive recurrent neural network | |
| CN109874029B (en) | Video description generation method, device, equipment and storage medium | |
| CN109376222B (en) | Question-answer matching degree calculation method, question-answer automatic matching method and device | |
| WO2022095380A1 (en) | Ai-based virtual interaction model generation method and apparatus, computer device and storage medium | |
| CN109857846B (en) | Method and device for matching user question and knowledge point | |
| CN111930931B (en) | Abstract evaluation method and device | |
| CN111368096A (en) | Knowledge graph-based information analysis method, device, equipment and storage medium | |
| CN113468891A (en) | Text processing method and device | |
| CN111695335A (en) | Intelligent interviewing method and device and terminal equipment | |
| CN117520503A (en) | Financial customer service dialogue generation method, device, equipment and medium based on LLM model | |
| CN110297897B (en) | Question-answer processing method and related product | |
| US11481609B2 (en) | Computationally efficient expressive output layers for neural networks | |
| CN113033912A (en) | Problem solving person recommendation method and device | |
| CN113705207A (en) | Grammar error recognition method and device | |
| CN110287487B (en) | Master predicate identification method, apparatus, device, and computer-readable storage medium | |
| CN116821324A (en) | Model training method and device, electronic equipment and storage medium | |
| CN115713065A (en) | Method for generating question, electronic equipment and computer readable storage medium | |
| CN112949305B (en) | Negative feedback information acquisition method, device, equipment and storage medium | |
| CN114757205A (en) | Text evaluation method, device, equipment and storage medium | |
| US20220277145A1 (en) | Domain Context Ellipsis Recovery for Chatbot | |
| CN111310460B (en) | Statement adjusting method and device | |
| CN111680515A (en) | Answer determination method and device based on AI (Artificial Intelligence) recognition, electronic equipment and medium | |
| CN110059314A (en) | A kind of Relation extraction method based on enhancing study | |
| CN111883111B (en) | Method, device, computer equipment and readable storage medium for processing speech training | |
| CN117932009A (en) | ChatGLM model-based insurance customer service dialogue generation method, chatGLM model-based insurance customer service dialogue generation device, chatGLM model-based insurance customer service dialogue generation equipment and ChatGLM model-based insurance customer service dialogue generation medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |