CN112434152B - Education choice question answering method and device based on multi-channel convolutional neural network - Google Patents

Abstract

The invention discloses a method and a device for solving elementary education choice questions based on a multi-channel convolutional neural network. The method comprises the following steps: 1) Giving a selection question presented in a text form, supplementing each option into an assertion, retrieving each assertion by using a subject knowledge base, and screening through a bridging rule to obtain a high-confidence evidence; 2) Processing the problem information and the high-confidence evidence by using a multi-channel convolutional neural network to obtain a confidence competition result between the options; 3) And judging the best option according to the confidence competition result among the options. The invention can retrieve high-confidence evidence from the discipline knowledge base by using a bridging attention mechanism, then simultaneously process the questions and the evidence by gating the multi-channel convolution neural network to obtain the comparison scores among the options, and further determine the best option based on the accumulated scores compared among all the option pairs, so that the machine can solve the specific discipline selection questions in the elementary education stage and obtain better performance.

Description

Method and device for answering multiple-choice questions in education based on multi-channel convolutional neural network

technical field

The invention belongs to the field of natural language question answering, and relates to a primary education multiple-choice question solver based on a multi-channel convolutional neural network. The solver can retrieve high-confidence evidence from the subject knowledge base using a bridged attention mechanism, and then process the question and evidence simultaneously through a gated multi-channel convolutional neural network to obtain a comparison score between options, which is then based on all options. The cumulative scores compared to determine the best option, allowing the machine to answer subject-specific multiple-choice questions in primary education and achieve better performance.

Background technique

With the development of machine learning and artificial intelligence technology, machines have achieved excellent performance in many natural language processing tasks, and even approach human performance in some tasks. Machine question answering is one of the rapidly developing fields. The machine question answering task requires the model to automatically answer the questions presented in the form of human natural language, which is one of the standards for measuring the language understanding ability of robots.

Multiple-choice question is an important question type used to comprehensively examine students' knowledge of various subjects in the primary education stage. Question instructions, select the most appropriate one from the candidates as the answer to the question. The multiple-choice questions in each subject examination paper in the primary education stage involve a wide range of knowledge, are difficult to answer, and have relatively fair evaluation criteria, which are very suitable for testing the natural language comprehension ability of machines. How to make machines perform better on multiple-choice questions in primary education has become an important topic in the field of natural language processing.

In order for machines to solve such multiple-choice questions, they are often processed using neural network techniques. Neural network is a widely used technology in natural language processing, which can extract high-level features in text through huge network structure. Convolutional neural network is a deep neural network that uses convolutional computing, which can model text information well and has stable and excellent performance. In the field of machine question answering, convolutional neural networks are often used for text feature extraction. However, the knowledge contained in the neural network is relatively limited. In order to make the model perform better in the knowledge-intensive question answering of specific disciplines, external knowledge and evidence such as termbases and knowledge bases are often introduced to assist the neural network.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a gated multi-channel convolutional neural network, combined with retrieval and bridging evidence screening mechanism, to better introduce the subject knowledge evidence into the neural network, so that the machine can obtain the multiple choice questions of primary education subjects. Methods and apparatus for better performance. That is, for a specific subject multiple-choice question in the primary education stage, high-confidence evidence can be retrieved from the subject knowledge base, and then the convolutional neural network is used to process the problem information and evidence, and the best answer can be obtained through the competition among the options.

In order to achieve the above object, the technical scheme of the present invention is:

A method for answering multiple-choice questions in primary education subjects based on a multi-channel convolutional neural network, comprising the following steps:

Given a multiple-choice question in text form (containing a question and multiple options), supplement each option as an assertion;

Use the subject knowledge base to retrieve each assertion, and filter the retrieved evidence through bridging rules to obtain high-confidence evidence;

Use multi-channel convolutional neural network to process problem information and high-confidence evidence, and obtain the results of confidence competition between options;

The best option is determined according to the confidence competition results among the options.

Further, given a multiple-choice question presented in the form of text, each option is supplemented as an assertion, including: for a question q and n options {op ₁ , op ₂ , ..., op _n } For multiple-choice questions, use rules to clean up the questions, and delete redundant expressions including "as shown in the figure" and "the correct option is". Concatenate the question q with the options {op ₁ , op ₂ , ..., op _n } to generate assertions {a ₁ , a ₂ , ..., a _n }. Mark the correctness of the assertion based on information such as the question reference answer and the negative words in the question.

Further, the retrieval of each assertion by using the subject knowledge base includes: according to the subject involved in the subject, collecting text information related to the subject from resources such as textbooks and online encyclopedia to construct a subject knowledge base K for retrieval, Provide evidence support for machines to answer multiple-choice questions. For each _assertion in _{ a ₁ , _{a 2} , . _m }.

Further, the bridging rule is used to screen the retrieved evidence to obtain high-confidence evidence, including: for each assertion retrieved _m pieces of evidence {k ₁ , k ₂ , . . . , km }, using The bridging mechanism scores each piece of evidence, and selects the l piece with the highest score as the high-confidence evidence {k′ ₁ , k′ ₂ , . . . , k′ _l } used later. For assertion a consisting of question q and option op, a word _wi in evidence k bridges the attention score as:

Among them, q _w and op _w are the word sets formed by question q and option op, respectively, and the cos function is used to calculate the cosine similarity of the word vectors of the two words. If the words in the questions and options appear repeatedly, each time they appear more than once, the calculated score is decreased by a certain rate (such as 0.9), so as to avoid giving too high scores to the words that appear many times. The score of the whole piece of evidence k is obtained by the weighted average of the scores of the t words with the highest score. The weighted calculation formula is:

为判断w_i的得分是否排名在前t内(比如t取值为5)的指示函数，pow(x，y)为求x的y次幂的函数。in,

In order to judge whether the score of _wi is ranked in the top t (for example, t is 5) the indicator function, pow(x, y) is the function of calculating x to the power of y.

For each assertion, the obtained l high-confidence evidences are combined to obtain the _{evidence {} e ₁ , e ₂ , . ≤n) is a set of supporting evidence for assertion a _i (1≤i≤n).

Further, after the bridging rule is used to screen the retrieved evidence, the data is segmented by a word segmentation method based on text matching (such as a forward maximum matching method) using a vocabulary containing corresponding subject terms. In order to reduce the complexity of the dataset and increase the data consistency, corresponding post-processing is performed on the unregistered words of different parts of speech.

Further, the multi-channel convolutional neural network is used to process question information and high-confidence evidence, and the confidence competition result between options is obtained. The gated multi-channel convolutional neural network model is used to process the question and evidence. For each group of options Score for comparison. Specifically, the network architecture is as follows:

1) In the embedding layer of the model, the question q, the two options op ₁ , op ₂ to be compared, and the corresponding assertions a ₁ , a ₂ and evidence e ₁ , e ₂ are encoded as word vectors. In this step, the vector representation can be a word vector trained by a neural language model (such as Word2vec, GloVe), a word vector obtained by reducing the dimension of a high-dimensional matrix by methods such as Singular Value Decomposition (SVD, Singular Value Decomposition) (such as potential Semantic analysis (LSA, Latent Semantic analysis) and other pre-trained low-dimensional dense semantic representation vectors can also be original high-dimensional sparse vectors such as One-hot Vectors.

2) In the convolutional layer of the model, the word vectors for question q, options op ₁ , op ₂ , assertions a ₁ , a ₂ and evidence e ₁ , e ₂ represent the convolutional neural network using multi-kernel and multi-step size at the same time in different processed in the channel. A multi-layer convolutional neural network can be used, and residual links can be taken. Residual connection refers to adding the output of the previous convolutional layer directly to the output of the current convolutional layer, replacing the output of the current layer for subsequent processing. If the two added tensor feature dimensions are different, usually Processed by a convolutional layer of width 1 for dimension adjustment. This link allows the deep convolutional network to adaptively adjust the network depth to a certain extent during the learning process, reducing the impact of the deep network on gradient propagation. Between layers, additional pooling layers (eg max pooling, average pooling) can be added to reduce the size of the feature matrix and pool the final output vector. Problem q, options op ₁ , op ₂ , processing of assertions a ₁ , a ₂ share one set of CNN parameters, and processing of evidence e ₁ , e ₂ uses another set of CNN parameters. After the pooling of the last convolutional layer, a gating mechanism is applied to the output by bitwise multiplication. Specifically, the outputs of options op ₁ and op ₂ are gated separately using the output of question q, the convolution output of evidence e ₁ is gated using the output of assertion a ₁ , and the evidence e is gated using the output of assertion a ₂ The output of ₂ is gated.

3) In the output layer of the model, for the four vector representations obtained in the previous step after the gating mechanism, the two vectors corresponding to each option are connected and then passed through the fully connected layer, and then the output vector of the fully connected layer is processed. After connecting, it passes through the fully connected layer again (where the output dimension of the last layer is 2), and finally gets the competition score of the two options.

Further, the best option is determined according to the result of the pairwise competition between the options obtained in the previous step. In the previous step, the competition score of option op _i (1≤i≤n) relative to option op _j (1≤j≤n, j≠i) is recorded as P(i>j), then the final cumulative comparison of option op _i The final _i score is:

For single-choice questions (that is, questions whose selection branches are directly expressed as answers), the option with the highest cumulative comparison score is selected as the best answer. For multiple-choice questions (that is, the question is given several answer statements first, and the selection branch is a question with some combination of numbers in these expressions), the cumulative comparison score of each numbered expression in each choice branch is accumulated, and the total score is calculated. The highest choice is considered the best answer.

The present invention also provides an apparatus for answering multiple-choice questions of elementary education based on a multi-channel convolutional neural network using the above method, which includes:

Assertion generation module for supplementing each option in a given multiple choice question presented in text form into an assertion;

Evidence retrieval and screening module is used to retrieve each assertion using the subject knowledge base, and filter the retrieved evidence through bridging rules to obtain high-confidence evidence;

A comparative scoring module is used to process question information and high-confidence evidence using multi-channel convolutional neural networks to obtain confidence competition results between options;

The best option judgment module is used for judging the best option according to the confidence competition result among the options.

Some modules in the above devices are not necessary, and they can still work normally after deleting or modifying some components. For example, after deleting the relevant components introduced by the evidence information (evidence retrieval and screening module and some components in the comparison scoring module), the device can still answer educational multiple-choice questions.

The beneficial effects of the present invention are as follows:

For a multiple-choice question of a specific subject in the primary education stage, the invention can retrieve high-confidence evidence from the subject knowledge base, and then use the convolutional neural network to process the question information and evidence, and obtain the best answer through the competition among the options. Achieve better performance on the multiple-choice questions in each subject at the education stage. The bridge attention mechanism used in the present invention can utilize the semantic relationship among questions, options and evidence to complete the screening of high-confidence evidence, thereby improving the correlation between the retrieved evidence and the topic. The multi-layer multi-channel convolutional neural network used in the present invention has strong representation ability and can extract the depth features of text from different angles. The gate control mechanism used in the present invention can introduce the retrieved evidence information into the neural network, complete the semantic interaction between questions and options, assertions and evidence, and finally make the model obtain better results in the multiple-choice questions of primary education subjects. Performance.

Description of drawings

FIG. 1 is a frame diagram of a method for answering multiple-choice questions of primary education in an embodiment of the present invention.

FIG. 2 is a frame diagram of a neural network in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It can be understood that the described embodiments are only a part of the embodiments of the present invention, rather than all the implementations. example. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

An example of the present invention is based on a data set composed of multiple-choice questions in a geography college entrance examination paper or a geography simulation test paper. It should be clearly understood by those skilled in the art that other candidate information sets and question sets may also be used in the specific implementation process.

Specifically, this example comes from 239 sets of geography examination papers, with a total of 4226 geography multiple-choice questions, each question is composed of questions and four options. Some questions contain graphs, but this method does not handle graphs.

As shown in FIG. 1 , it is a frame diagram of a method for answering multiple-choice questions of elementary education in an embodiment of the present invention; as shown in FIG. 2 , it is a framework diagram of a neural network in an embodiment of the present invention. Specific steps are as follows:

Step 1: Perform preprocessing steps such as assertion generation, evidence retrieval, attention mechanism screening, and unregistered word processing for the multiple-choice data set;

Specifically, use rules to clean up problems, and delete redundant expressions such as "as shown in the figure" and "the correct option is". Connect the question to each option to generate an assertion. Mark the correctness of the assertion based on information such as the question reference answer and the negative words in the question.

Collect required knowledge texts from geography textbooks, Baidu Encyclopedia and Wikipedia related pages, and use Lucene to build a subject knowledge base. For each assertion, after the retrieval produces 50 retrieval results, the bridge attention mechanism is used to score and reorder, and the top 5 results are taken as high-confidence evidence.

Combined with the geography vocabulary, the data is segmented by the forward maximum matching method, and then post-processing is performed: the unregistered words of NN and NR parts of speech (common nouns, proper nouns) are returned to special numbers, and punctuation and words that do not contain Chinese characters are returned. Unregistered words are assigned special numbers, and the unregistered words of NT and CD parts of speech (time nouns, numerals) are returned to the part-of-speech number, and the word vector of the part-of-speech number is set to a randomly initialized vector.

Step 2: Process the question and evidence using a gated multi-channel convolutional neural network model to get a competition score for each two options.

In the embedding layer of the model, word embeddings for questions, options, assertions, and evidences are performed using pretrained word vectors.

In the convolutional layer of the model, a two-layer convolutional neural network is used to perform multi-channel processing on the word embedding representation of questions, options, assertions and evidence. There are a total of 1280 convolution kernels, of which 512 are 100*1 convolution kernels. , 512 convolution kernels of size 100*2, and 256 convolution kernels of size 100*3. The CNNs for the question, options, and assertion channels share parameters, and the CNNs for the evidence channel use a different set of parameters. Residual links are used between the two layers of convolutional neural networks, and max pooling is used in both pooling layers. The question and assertion channels in the convolutional layer are activated by the sigmoid function before pooling. The gating mechanism uses a bitwise multiplication method, using the output representation of the question to gate the output representation of the two options separately, and using the output representation of the assertion to gate the output representation of the corresponding evidence.

In the output layer, the two vectors corresponding to each option are connected and passed through the fully connected layer, and then the output vector of the fully connected layer is connected and then passed through the fully connected layer again, and finally the comparison score between the two options is obtained. Specifically, the output vectors of the two gating mechanisms corresponding to each option are first connected, and the obtained 2560-dimensional vector passes through a fully connected network with an output dimension of 512, and then the two output 512-dimensional vectors are connected. Pass through the fully connected network with output dimensions of 1024 and 2 in turn. The 50% Dropout mechanism is used in the above fully connected layers to prevent overfitting, and the activation function uses ReLU.

Step 3: According to the comparison scores between the options obtained in the previous step, use the following formula to calculate the final score of each option, and select the option with the highest score as the final answer. In the previous step, the competition score of option op _i (1≤i≤n) relative to option op _j (1≤j≤n, j≠i) is recorded as P(i>j), then the final cumulative comparison of option op _i The final _i score is:

For single-choice questions (that is, questions whose selection branches are directly expressed as answers), the option with the highest cumulative comparison score is selected as the best answer. For multiple-choice questions (that is, the question is given several answer statements first, and the selection branch is a question with some combination of numbers in these expressions), the cumulative comparison score of each numbered expression in each choice branch is accumulated, and the total score is calculated. The highest choice is considered the best answer.

For the college entrance examination geographic multiple-choice question dataset used, the accuracy rate (Accuracy) is reported, that is, the frequency with which the model predicts that the option with the highest score is the standard answer. The model effect is shown in Table 1. Among them, if there is no picture title for the original title, there is no title with a picture, and there is a picture title for the original title with a picture. The model answers based on textual information, and there is no special module to deal with the information in the picture. For these originally pictured problems, we assume that there is no picture, and also use it for model training.

Table 1: The effect of the solver on the geography multiple-choice questions of the college entrance examination

Overall, it can be seen that the performance of the models under both settings has a large improvement over the baseline method of randomly selecting answers (25% accuracy).

Another embodiment of the present invention provides an apparatus for answering multiple-choice questions in elementary education based on a multi-channel convolutional neural network using the method of the present invention, which includes:

Assertion generation module for supplementing each option in a given multiple choice question presented in text form into an assertion;

Evidence retrieval and screening module is used to retrieve each assertion using the subject knowledge base, and filter the retrieved evidence through bridging rules to obtain high-confidence evidence;

A comparative scoring module is used to process question information and high-confidence evidence using multi-channel convolutional neural networks to obtain confidence competition results between options;

The best option judgment module is used for judging the best option according to the confidence competition result among the options.

For the specific implementation of the above modules, refer to the foregoing description of the method of the present invention.

Another embodiment of the present invention provides an electronic device (computer, server, etc.) comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer The program includes instructions for carrying out the steps in the method of the present invention.

Another embodiment of the present invention provides a computer-readable storage medium (eg, ROM/RAM, magnetic disk, optical disk), where the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method of the present invention is implemented of the various steps.

The parts of the present invention that are not described in detail belong to the well-known technology of those skilled in the art.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (8)

1. a method for answering multiple-choice questions of education based on multi-channel convolutional neural network, is characterized in that, comprises the following steps: Given a multiple choice question presented in text form, supplement each option as an assertion; Use the subject knowledge base to retrieve each assertion, and filter the retrieved evidence through bridging rules to obtain high-confidence evidence; Use multi-channel convolutional neural network to process problem information and high-confidence evidence, and obtain the results of confidence competition between options; Determine the best option according to the confidence competition results between options; The subject knowledge base is used to retrieve each assertion, and the retrieved evidence is screened through bridging rules to obtain high-confidence evidence, including: For each assertion, retrieve _m pieces of evidence {k ₁ ,k ₂ ,...,km } whose similarity of text meets the set threshold from the subject knowledge base; use bridging mechanism for each piece of evidence in the m pieces of evidence. Score, and select l with the highest score as high-confidence evidence {k′ ₁ ,k′ ₂ ,...,k _l ′}; for assertion a composed of question q and option op, one of the evidence k The bridge attention score for word _wi is:

Among them, q _w and op _w are the word sets formed by question q and option op, respectively, and the cos function is used to calculate the cosine similarity of the word vectors of the two words; if the words in the question and the option appear repeatedly, each time they appear more Once, the calculated score is decreased at a rate of 0.9; the score of the entire piece of evidence k is obtained by the weighted average of the scores of the highest scoring t words. The weighted calculation formula is:

in,

In order to judge whether the score of _wi is ranked in the top t, pow(x, y) is the function of finding the power of y of x; For each assertion, the obtained l high-confidence evidences are combined to obtain the evidence {e ₁ ,e ₂ ,..., _en } for each assertion to be input to the neural network, where e _i is one of the assertions a _i Group supporting evidence, 1≤i≤n; The multi-channel convolutional neural network is used to process the problem information and high-confidence evidence, and the confidence competition results between the options are obtained, including: In the embedding layer of the model, each word in the question q, the two options to be compared op ₁ , op ₂ and the corresponding assertions a ₁ , a ₂ and evidence e ₁ , e ₂ are encoded as word vectors; In the convolutional layer of the model, the word vector representation for question q, options op ₁ , op ₂ , assertions a ₁ , a ₂ and evidence e ₁ , e ₂ uses a multi-kernel and multi-step convolutional neural network simultaneously in different channels process in the process; multi-layer convolutional neural network is used for processing, and residual link is adopted, additional pooling layer is added between layers to reduce the scale of feature matrix, and the final output vector is pooled; Question q, options op ₁ , op ₂ , the processing of assertions a ₁ , a ₂ share a set of CNN parameters, and the processing of evidence e ₁ , e ₂ uses another set of CNN parameters; After the pooling of the product layer, the gating mechanism of bitwise multiplication is performed on the output; In the output layer of the model, for the four vector representations after the gating mechanism, the two vectors corresponding to each option are connected and then passed through the fully connected layer, and then the output vectors of the fully connected layer are connected and then passed through the fully connected layer again , and finally get a competitive score for both options. 2. The method for answering educational multiple-choice questions based on a multi-channel convolutional neural network as claimed in claim 1, wherein the given multiple-choice questions presented in text form supplement each option as an assertion, including : Use rules to clean up questions and delete redundant expressions including "as shown in the figure" and "the correct option is"; connect questions with various options to generate assertions; refer to answers and negative word information in questions to mark assertions correctness. 3. The method for answering multiple-choice questions in education based on multi-channel convolutional neural networks as claimed in claim 1, wherein the method for constructing the subject knowledge base is: according to the subject involved in the subject, from textbooks and network resources Collect text information related to the subject and build a subject knowledge base for retrieval. 4. The method for answering educational multiple-choice questions based on a multi-channel convolutional neural network according to claim 1, characterized in that, after the bridging rule is used to screen the retrieved evidence, words containing corresponding subject terms are used. The table uses the word segmentation method based on text matching for word segmentation, and performs corresponding post-processing on unregistered words with different parts of speech and words with special parts of speech. 5. The method for answering educational multiple-choice questions based on a multi-channel convolutional neural network as claimed in claim 1, wherein the best option is determined according to the confidence competition result between the options, comprising: The competition score of option op _i relative to option op _j is recorded as P(i>j), where 1≤i≤n, 1≤j≤n, j≠i, where n represents the number of options; then the final result of option op _i The cumulative comparison score final _i is:

For single-choice questions, the option with the highest cumulative comparison score is selected as the best answer; for multiple-choice questions, the cumulative comparison scores of the expressions of each number in each choice branch are accumulated, and the choice branch with the highest total score is regarded as the best answer. Best answer. 6. A device for answering educational multiple-choice questions based on a multi-channel convolutional neural network using the method according to any one of claims 1 to 5, characterized in that, comprising: Assertion generation module for supplementing each option in a given multiple choice question presented in text form into an assertion; Evidence retrieval and screening module is used to retrieve each assertion using the subject knowledge base, and filter the retrieved evidence through bridging rules to obtain high-confidence evidence; A comparative scoring module is used to process question information and high-confidence evidence using multi-channel convolutional neural networks to obtain confidence competition results between options; The best option judgment module is used for judging the best option according to the confidence competition result among the options. 7. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes a program for executing claims 1- 5. Instructions for each step in the method for solving educational multiple-choice questions based on a multi-channel convolutional neural network according to any one of claims. 8. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the multi-based multi-based system according to any one of claims 1 to 5 is implemented. An educational multiple-choice problem-solving method for channel convolutional neural networks.

Images