CN114969260A - Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning - Google Patents

Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning Download PDF

Info

Publication number
CN114969260A
CN114969260A CN202210597773.7A CN202210597773A CN114969260A CN 114969260 A CN114969260 A CN 114969260A CN 202210597773 A CN202210597773 A CN 202210597773A CN 114969260 A CN114969260 A CN 114969260A
Authority
CN
China
Prior art keywords
test
questions
test question
subjective
answer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210597773.7A
Other languages
Chinese (zh)
Inventor
罗建华
陈意山
张兰芳
龚云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin Tourism University
Original Assignee
Guilin Tourism University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin Tourism University filed Critical Guilin Tourism University
Priority to CN202210597773.7A priority Critical patent/CN114969260A/en
Publication of CN114969260A publication Critical patent/CN114969260A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention discloses an automatic subjective question evaluation deep learning method combining test question classification and evaluation learning, and relates to the crossing field of artificial intelligence and intelligent education. According to the invention, aiming at the characteristics that different types of subjective questions have different evaluation modes, the automatic evaluation neural network model for the subjective questions can master the evaluation method for the different types of the test questions through the combined test question classification and evaluation learning, so that the automatic evaluation problem of the subjective questions is solved by a more effective method. Firstly, the subjective questions to be evaluated are classified through an integrating neural network, and the types of the subjective questions to be evaluated are obtained. Then, the test question type embedding of the subjective questions to be evaluated is connected with each character embedding of the student answer input sequence and the standard answer input sequence to obtain student answer input and standard answer input of test question type perception, and high-precision scoring is carried out through a heterozygous model consisting of convolution and a bidirectional long-short term memory network.

Description

Subjective question automatic evaluation and reading deep learning method combining test question classification and evaluation learning
Technical Field
The invention relates to the crossing field of artificial intelligence and intelligent education, in particular to a subjective question automatic scoring deep learning method combining test question classification and scoring learning, which can be widely applied to a subjective question computer automatic scoring system of each subject.
Background
The examination questions in the examination paper are generally divided into two categories, objective questions and subjective questions, in terms of the form of answer composition. The test questions such as the single-choice question, the multiple-choice question, and the judgment question, whose answers are expressed by the choice numbers, are called objective questions, and the test questions such as the short answer question, the noun explanation question, and the discussion question, whose answers are expressed by natural language, are called subjective questions. Because the answers of objective questions such as single-choice questions, multiple-choice questions, judgment questions and the like are all expressed by option numbers, when the current computer automatically scores the questions, only simple matching operation needs to be carried out on the option numbers of the standard answers and the option numbers of the student answers, and the answers are correct if the matching is successful, so that the processing technology has achieved better results. However, the automatic scoring technology of the subjective questions with answers expressed by natural language is as follows: for automatic examination paper of simple answer questions, noun explanation and discussion questions and the like, the effect is not ideal because the examination paper is influenced by theories and technical bottlenecks such as natural language understanding, artificial intelligence and the like.
With the continuous development of artificial neural network technology, many deep learning models such as LSTM-based models, CNN & LSTM-based models, and transform-based models are applied to subjective question review. These deep learning models automatically extract semantic features from the answer text using different neural networks, thereby providing an end-to-end approach that does not require any artificial feature engineering. However, the deep learning method of automatic subjective question review still has many challenges. Among these, one important challenge is: the answers of some question types of the subjective questions have the orderliness, but the answers of some question types of the subjective questions do not have the orderliness, for example, for the test question, "please write the units of the storage capacity in sequence from small to large", the answer key points "B, KB, MB, and GB" have the orderliness; however, for the test question "brief description of the advantages of twisted pair", the answer key points "high flexibility, oil resistance, water resistance, wear resistance, acid resistance, cold resistance, bending resistance" have no sequence, that is, the answer of the student "water resistance, cold resistance, bending resistance, oil resistance, wear resistance, acid resistance, high flexibility" is also completely correct. Therefore, how to automatically learn different review methods for different question types is one of the problems to be solved by the subjective question automatic review deep learning method.
Disclosure of Invention
The invention discloses an automatic subjective question evaluation deep learning method combining test question classification and grading learning.
In order to achieve the purpose, the technical scheme of the invention is as follows:
a subjective question automatic evaluation deep learning method combining test question classification and evaluation learning is characterized by comprising the following steps:
s1, classifying the test questions of the subjective questions to be evaluated through a CNN test question classification submodel with residual connection to obtain the types of the test questions of the subjective questions to be evaluated;
s2, embedding and connecting the test question type of the subjective question to be evaluated with each character of the student answer input sequence and the standard answer input sequence to obtain student answer perception input and standard answer perception input of test question type perception, and then sending the student answer perception input and the standard answer perception input into a subjective question test paper grading sub-model consisting of CNN and Bi-LSTM to grade to obtain the grading grade of the student answer;
s3, training a subjective question automatic evaluation neural network model for combined test question classification and evaluation learning through the loss functions of the combined calculation step S1 and the step S2;
s4, automatically evaluating the neural network model by using the formed subjective questions trained in the step S3, and evaluating any student answers;
CNN is an abbreviation for the English conditional Neural Networks, representing a Convolutional Neural network, and is described by the documents "Lawrence S, Giles CL, Tsio AC et al. face repetition: A conditional Neural Networks, IEEE Transactions on 1997; 8(1) 98-113';
the Bi-LSTM is an abbreviation of English Directional Long Short-Term Memory and represents a two-way Long-Short Term Memory Neural network, wherein the LSTM Neural network is proposed by the documents "Sepp Hochreiter and Juregen Schmidhuber.1997 Long Short-Term Memory, 9(8): 1735-;
the subjective question types comprise definition subjective question test questions, sequence subjective question test questions and general subjective question test questions, wherein the definition subjective question test questions refer to subjective questions requiring students to explain definition of a certain concept, the sequence subjective question test questions refer to subjective questions with answers including a plurality of key points given in sequence, and the general subjective question test questions refer to other subjective questions except the definition and the sequence.
Further, the step S1 specifically includes:
s1.1 character sequence of test question to be classified
Figure BDA0003668781920000021
Word embedding sequence T converted into test questions to be classified through table look-up operation e The calculation process is as follows:
Figure BDA0003668781920000022
Figure BDA0003668781920000023
wherein n represents the sentence length of the test question to be classified, onehot (T) represents the conversion of the test question word sequence T into a unique heat matrix, T o Is a one-hot matrix of the test word sequence T, | V | represents the size of a dictionary V, which is a dictionary established by a model and containing all words,
Figure BDA0003668781920000031
a matrix is embedded for the pre-trained words corresponding to dictionary V,
Figure BDA0003668781920000032
representing a word embedding vector corresponding to the ith word in the test question word sequence T, and h representing the dimension of the word embedding vector;
s1.2 embedding words of test questions to be classified into sequence T e Inputting the data into a convolutional neural network adopting zero filling to obtain shallow semantic representation T of the test question to be classified c The calculation process is as follows:
Figure BDA0003668781920000033
Figure BDA0003668781920000034
wherein PCNN (-) represents a convolutional neural network with zero padding, (T) c ) ij Represents T c The ith row and the jth column of (c), k is the convolution kernel size, d c Is the number of the convolution kernels and is,
Figure BDA0003668781920000035
word-embedded sequence T representing test questions to be classified e As a result of the zero-padding being performed,
Figure BDA0003668781920000036
to represent
Figure BDA0003668781920000037
The j (th) column to the j + (k-1) column of the matrix,
Figure BDA0003668781920000038
the weight of the ith convolution kernel is represented,
Figure BDA0003668781920000039
is shown asThe offset of the i convolution kernels is such that,
Figure BDA00036687819200000310
representing a convolution operation;
s1.3 shallow semantic representation T of test questions to be classified c Sending the test question into a feedforward neural network consisting of two full-connection layers for enhancement to obtain enhanced semantic representation of the test question to be classified
Figure BDA00036687819200000311
The calculation process is as follows:
Figure BDA00036687819200000312
wherein the content of the first and second substances,
Figure BDA00036687819200000313
respectively the weight matrices of the two fully-connected layers,
Figure BDA00036687819200000314
bias vectors of two fully-connected layers respectively, ReLU is an abbreviation of English 'Rectified Linear Unit', and ReLU (·) represents an activation function adopting modified Linear Unit operation;
s1.4, representing T by shallow semantics of test questions to be classified c With its enhanced semantic representation T f Residual error connection is carried out to obtain deep semantic representation T of the test question to be classified u The calculation process is as follows:
Figure BDA00036687819200000315
wherein LayerNorm (·) denotes layer normalization;
s1.5 deep semantic representation T of test questions to be classified u Performing maximum pooling to obtain a classification vector Z of the test questions to be classified c The calculation process is as follows:
Figure BDA00036687819200000316
wherein maxPooling (·) represents a max pooling operation;
s1.6 Classification vector Z of test questions to be classified c Executing linear transformation of softmax, performing probability calculation of the test question types, and obtaining the final test question types, wherein the calculation process is as follows:
o＝M c Z c +b o (8)
Figure BDA0003668781920000041
Figure BDA0003668781920000042
wherein the content of the first and second substances,
Figure BDA0003668781920000043
is a matrix of the test question type representation,
Figure BDA0003668781920000044
is an offset vector, d y Is the number of the categories of the test question types,
Figure BDA0003668781920000045
represents Z c Confidence score vectors over all test question types, y being one test question type, o y Represents Z c Confidence score, o, labeled test question type y i Represents Z c Confidence score, Ρ (yz | Z) on the ith test question type c ) Classification vector Z representing test questions to be classified c The prediction probability on the question type Y, which is the set of all question types,
Figure BDA0003668781920000046
the representation returns such that Pp (y | Z) c ) Test question type of maximum value, y test question type of final classification of test questions to be classified, exp(. cndot.) represents an exponential function based on a natural constant e.
Further, the step S2 specifically includes:
s2.1, performing table look-up operation on the student answer word sequence and the standard answer word sequence according to the step S1.1 to respectively obtain student answer word embedded sequences
Figure BDA0003668781920000047
Standard answer word embedding sequence
Figure BDA0003668781920000048
Then embedding the test words into the sequence T e Respectively embedding the answer words with students
Figure BDA0003668781920000049
Standard answer word embedding sequence
Figure BDA00036687819200000410
Splicing the head and the tail to obtain an answer input sequence A of the student e And a standard answer input sequence S e The calculation process is as follows:
Figure BDA00036687819200000411
Figure BDA00036687819200000412
wherein, ": "denotes the end-to-end concatenation operation of the sequence, n1 is the sentence length of student answers, n2 is the sentence length of standard answers, m1 is the student answer input sequence A e M1 ═ n + n1, m2 is the standard answer input sequence S e And m2 ═ n + n 2;
s2.2 converting the test question type y obtained in the step S1 into the corresponding test question type embedding through table lookup operation
Figure BDA00036687819200000413
Then v is measured t Respectively input with student's answer input sequence A e And a standard answer input sequence S e Each character in the Chinese character string is embedded and connected to obtain student answer perception input A t And standard answer perception input S t The calculation process is as follows:
Figure BDA0003668781920000051
Figure BDA0003668781920000052
wherein the content of the first and second substances,
Figure BDA0003668781920000053
respectively, student's answer input sequence A e And student answer perception input A t Word-embedding vector of the ith column, d t Is the dimension of the embedding of the test question types,
Figure BDA0003668781920000054
respectively, standard answer input sequence S e And standard answer perception input S t The word in column j embeds a vector, "; "represents a concatenation operation of vectors;
s2.3 inputting student answer perception A t And standard answer perception input S t Respectively sending the data to two convolutional neural networks which adopt zero padding and share parameters to obtain shallow semantic representation A of student answers c And a standard answer shallow semantic representation S c The calculation process is as follows:
Figure BDA0003668781920000055
Figure BDA0003668781920000056
wherein sharedPCNN1(·) and sharedPCNN2 denote that two take zeroConvolutional neural networks populated and sharing parameters, d c The number of convolution kernels;
s2.4 shallow semantic representation A of student answers c And the standard answer shallow semantic representation S c Sending the data into an interactive network consisting of Bi-LSTM to obtain a scoring vector Z of student answers g As follows:
Figure BDA0003668781920000057
wherein Interactive (-) represents an Interactive network composed of Bi-LSTM, d L Dimension of hidden state of Bi-LSTM in interactive network;
s2.5 Scoring vector Z to student' S answer g Performing linear transformation of softmax, performing probability calculation of the grade, and obtaining a final grade, wherein the calculation process is as follows:
x＝M g Z g +b g (18)
Figure BDA0003668781920000058
Figure BDA0003668781920000059
wherein the content of the first and second substances,
Figure BDA00036687819200000510
is a matrix of the score level representation,
Figure BDA00036687819200000511
is an offset vector, d g Is the number of the rating levels to be scored,
Figure BDA0003668781920000061
is represented by Z g Vector of confidence scores of all the rating levels, g being one rating level, x g Represents Z g Marked as score level gConfidence score, x i Represents Z g Confidence score on the i-th scoring level, Ρ (g | Z) g ) Representing a score vector Z g The prediction probability on the score scale G, G being the set of all score scales, G being the final rated score scale,
Figure BDA0003668781920000062
denotes return such that Pp (g | Z) g ) The score rating of the maximum value.
Further, the step S2.4 specifically includes:
s2.4.1 use Bi-LSTM of two shared parameters to respectively represent A to the shallow semantic of student answer c And the standard answer shallow semantic representation S c Encoding to obtain student answer context expression A r And a standard answer context representation S r The calculation process is as follows:
Figure BDA0003668781920000063
Figure BDA0003668781920000064
wherein ShardedBilSTM 1 (-), ShardedBilSTM 2 (-), represent Bi-LSTM sharing two parameters,
Figure BDA0003668781920000065
respectively represent A c And S c The connection result of the Bi-directional hidden state corresponding to the ith time step in the Bi-LSTM,
Figure BDA0003668781920000066
respectively represent A c And S c From left to right
Figure BDA0003668781920000067
Hidden state at the ith time step in the network,
Figure BDA0003668781920000068
respectively represent A c And S c In a right to left direction
Figure BDA0003668781920000069
Hidden state at ith time step in the network;
s2.4.2 context representation of student answers by attention mechanism A r And a standard answer context representation S r Interact to get mutual attention
Figure BDA00036687819200000610
And
Figure BDA00036687819200000611
the calculation process is as follows;
Figure BDA00036687819200000612
Figure BDA00036687819200000613
Figure BDA00036687819200000614
Figure BDA00036687819200000615
Figure BDA00036687819200000616
Figure BDA00036687819200000617
wherein the content of the first and second substances,
Figure BDA00036687819200000618
is composed of
Figure BDA00036687819200000619
The (c) th element of (a),
Figure BDA00036687819200000620
the (c) th element of (a),
Figure BDA00036687819200000621
respectively represent pair
Figure BDA0003668781920000071
And
Figure BDA0003668781920000072
the transposition operation is carried out and the operation is carried out,
Figure BDA00036687819200000724
to refer to a logical quantity of "all",
Figure BDA0003668781920000073
respectively representing hidden state tuples
Figure BDA0003668781920000074
And
Figure BDA0003668781920000075
attention weight of (1);
s2.4.3 respectively will be
Figure BDA0003668781920000076
And
Figure BDA0003668781920000077
with student answer context representation A r And a standard answer context representation S r Fusing to obtain student answer fusion expression A f Fused representation S with standard answers f The calculation process is as follows:
Figure BDA0003668781920000078
Figure BDA0003668781920000079
wherein the content of the first and second substances,
Figure BDA00036687819200000710
is A f The (c) th element of (a),
Figure BDA00036687819200000711
is S f The ith element in (1), Fusion (-), represents a Fusion operation, and an element-by-element multiplication operation of the matrix;
s2.4.4 fusion of student answers using two shared parameters Bi-LSTM respectively f Fused representation S with standard answers f Performing enhancement processing to obtain deep semantic representation A of student answers u And standard answer deep semantic representation S u The calculation process is as follows:
Figure BDA00036687819200000712
Figure BDA00036687819200000713
wherein the content of the first and second substances,
Figure BDA00036687819200000714
respectively represent A f And S f The connection result of the Bi-directional hidden state corresponding to the ith and jth time steps in the Bi-LSTM;
s2.4.5 deep semantic representation A of student answers respectively u And standard answer deep semantic representation S u Carrying out average pooling and maximum pooling, and connecting all pooled vectors to obtain score vector Z of student answer g The calculation process is as follows:
Figure BDA00036687819200000715
Figure BDA00036687819200000716
Figure BDA00036687819200000717
Figure BDA00036687819200000718
Figure BDA00036687819200000719
wherein maxPooling (. cndot.) represents the maximum pooling operation, avePooling (. cndot.) represents the average pooling operation,
Figure BDA00036687819200000720
is A u The maximum pooling vector of (a) is,
Figure BDA00036687819200000721
is A u The average pooling vector of (a) is,
Figure BDA00036687819200000722
is S u The maximum pooling vector of (a) is,
Figure BDA00036687819200000723
is S u The average pooling vector of "; "represents a concatenation operation of vectors.
Further, the step S3 specifically includes:
s3.1, calculating a loss function of the test question classification submodel and a loss function of the subjective question automatic scoring submodel by using the cross entropy loss error respectively, wherein the calculation process is as follows:
Figure BDA0003668781920000081
Figure BDA0003668781920000082
wherein, omega is a training set of the subjective question automatic evaluation task of the joint test question classification and evaluation learning, | omega | represents the size of the training set omega,
Figure BDA0003668781920000083
is the true test question classification of the ith training sample in omega,
Figure BDA0003668781920000084
is the true score level of the ith training sample in omega,
Figure BDA0003668781920000085
is the classification vector of the test question of the ith training sample in omega,
Figure BDA0003668781920000086
is the score vector of student answer of the ith training sample in omega, gamma 1 Is a loss function used in the training of the test question classification submodel, gamma 2 Is a loss function used when carrying out the automatic scoring submodel training of the subjective questions;
s3.2, calculating a joint loss function of the subjective question automatic evaluation neural network model for joint test question classification and evaluation learning by using the following formula
Figure BDA0003668781920000087
Figure BDA0003668781920000088
Where λ and β are two weight parameters;
s3.3 the training goal of the neural network model for automatic review of subjective questions in combination of test question classification and score learning is to minimize the combined loss error of the formula (40).
Further, the table lookup operation in step S2.2 specifically includes:
Figure BDA0003668781920000089
Figure BDA00036687819200000810
wherein, OneHot (y) * ) Label y for indicating to-be-tested question type * Is converted into a one-hot vector,
Figure BDA00036687819200000811
for test question type y * The one-hot vector of (a) is,
Figure BDA00036687819200000812
embedding matrices for question types, d y Is the number of categories of test question types, d t Is the dimension of the question type embedding.
The invention provides an automatic subjective question evaluation deep learning method combining test question classification and grading learning aiming at the characteristic that different types of subjective questions have different evaluation modes. Firstly, classifying the subjective questions to be evaluated through a CNN test question classification submodel with residual connection to obtain the types of the subjective questions to be evaluated. Then, the test question type embedding of the subjective questions to be evaluated is connected with each character embedding of the student answer input sequence and the standard answer input sequence to obtain student answer perception input and standard answer perception input of test question type perception, and the examination paper is evaluated on the basis.
The invention has the following advantages:
(1) classifying the test questions of the subjective questions to be evaluated through a CNN test question classification submodel with residual connection, so that the model can master the types of the test questions;
(2) embedding and connecting the test question type of the subjective question to be evaluated with each character of the student answer input sequence and the standard answer input sequence, so that the grading sub-module can sense the test question type;
(3) the test question classification and the test question scoring are jointly learned, so that the precision of the test question classification and the adaptivity of the test question scoring can be further improved;
(4) the scoring is carried out through a hybrid neural network consisting of a convolutional neural network and a bidirectional long-short term memory neural network, so that the scoring precision can be greatly improved.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a schematic diagram of the Bi-LSTM interaction network of the present invention.
Detailed Description
The present invention is further illustrated by the following specific examples, but the scope of the present invention is not limited to the following examples.
According to the method flow chart shown in fig. 1, the automatic subjective question evaluation deep learning method combining test question classification and evaluation learning is realized through the following steps:
s1, classifying the test questions of the subjective questions to be evaluated through a CNN test question classification submodel with residual connection to obtain the types of the test questions of the subjective questions to be evaluated;
s2, embedding and connecting the test question type of the subjective question to be evaluated with each character of the student answer input sequence and the standard answer input sequence to obtain student answer perception input and standard answer perception input of test question type perception, and then sending the student answer perception input and the standard answer perception input into a subjective question test paper grading sub-model consisting of CNN and Bi-LSTM to grade to obtain the grading grade of the student answer;
s3, training a subjective question automatic evaluation neural network model for combined test question classification and evaluation learning through the loss functions of the combined calculation step S1 and the step S2;
s4, automatically evaluating the neural network model by using the formed subjective questions trained in the step S3, and evaluating any student answers;
CNN is an abbreviation of the English conditional Neural Networks, representing a Convolutional Neural network, and is known from the literature "Lawrence S, Giles CL, Tsio AC et al. face recognition: A conditional Neural-network approach. Neural Networks, IEEE Transactions on 1997; 8(1) 98-113';
the Bi-LSTM is an abbreviation of English Directional Long Short-Term Memory and represents a two-way Long-Short Term Memory Neural network, wherein the LSTM Neural network is proposed by the documents "Sepp Hochreiter and Juregen Schmidhuber.1997 Long Short-Term Memory, 9(8): 1735-;
the subjective question types comprise definition subjective question test questions, sequence subjective question test questions and general subjective question test questions, wherein the definition subjective question test questions refer to subjective questions requiring students to explain definition of a certain concept, the sequence subjective question test questions refer to subjective questions with answers comprising a plurality of key points given in sequence, and the general subjective question test questions refer to other subjective questions except the definition and the sequence.
Further, the step S1 specifically includes:
s1.1 character sequence of test question to be classified
Figure BDA0003668781920000101
Word embedding sequence T converted into test question to be classified through table look-up operation e The calculation process is as follows:
Figure BDA0003668781920000102
Figure BDA0003668781920000103
wherein n represents the sentence length of the test question to be classified, onehot (T) represents the conversion of the test question word sequence T into a unique heat matrix, T o Is a one-hot matrix of the test word sequence T, | V | represents the size of a dictionary V, which is a dictionary established by a model and containing all words,
Figure BDA0003668781920000104
a matrix is embedded for the pre-trained words corresponding to dictionary V,
Figure BDA0003668781920000105
representing a word embedding vector corresponding to the ith word in the test question word sequence T, and h representing the dimension of the word embedding vector;
s1.2 embedding words of test questions to be classified into sequence T e Inputting the data into a convolutional neural network adopting zero filling to obtain shallow semantic representation T of the test question to be classified c The calculation process is as follows:
Figure BDA0003668781920000106
Figure BDA0003668781920000107
wherein PCNN (. cndot.) represents a convolutional neural network with zero padding, (T) c ) ij Represents T c The ith row and the jth column of (c), k is the convolution kernel size, d c Is the number of the convolution kernels and is,
Figure BDA0003668781920000111
word-embedded sequence T representing test questions to be classified e As a result of the zero-padding being performed,
Figure BDA0003668781920000112
to represent
Figure BDA0003668781920000113
The j (th) column to the j + (k-1) column of the matrix,
Figure BDA0003668781920000114
the weight of the ith convolution kernel is represented,
Figure BDA0003668781920000115
represents the offset of the ith convolution kernel,
Figure BDA0003668781920000116
representing a convolution operation;
s1.3 shallow semantic representation T of test questions to be classified c Sending the test question into a feedforward neural network consisting of two full-connection layers for enhancement to obtain enhanced semantic representation of the test question to be classified
Figure BDA0003668781920000117
The calculation process is as follows:
Figure BDA0003668781920000118
wherein the content of the first and second substances,
Figure BDA0003668781920000119
respectively the weight matrices of the two fully-connected layers,
Figure BDA00036687819200001110
bias vectors of two fully-connected layers respectively, ReLU is an abbreviation of English 'Rectified Linear Unit', and ReLU (·) represents an activation function adopting modified Linear Unit operation;
s1.4, expressing T by shallow semantic of test questions to be classified c With its enhanced semantic representation T f Residual error connection is carried out to obtain deep semantic representation T of the test question to be classified u The calculation process is as follows:
Figure BDA00036687819200001111
wherein LayerNorm (·) denotes layer normalization;
s1.5 deep semantic representation T of test questions to be classified u Performing maximum pooling to obtain a classification vector Z of the test questions to be classified c The calculation process is as follows:
Figure BDA00036687819200001112
wherein maxPooling (·) represents a max pooling operation;
s1.6 Classification vector Z of test questions to be classified c Executing linear transformation of softmax, performing probability calculation of the test question types, and obtaining the final test question types, wherein the calculation process is as follows:
o＝M c Z c +b o (8)
Figure BDA00036687819200001113
Figure BDA00036687819200001114
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA00036687819200001115
is a matrix of the test question type representation,
Figure BDA00036687819200001116
is an offset vector, d y Is the number of the categories of the test question types,
Figure BDA0003668781920000121
represents Z c Confidence score vectors over all test question types, y being one test question type, o y Represents Z c Confidence score, o, labeled test question type y i Represents Z c Confidence score, Ρ (yz | Z) on the ith test question type c ) Classification vector Z representing test questions to be classified c In trialThe prediction probability on topic type Y, which is the set of all question types,
Figure BDA0003668781920000122
the representation returns such that Pp (y | Z) c ) Type of question at maximum, y * For the finally classified test question type of the test questions to be classified, exp (-) represents an exponential function with a natural constant e as the base.
Further, the step S2 specifically includes:
s2.1, performing table look-up operation on the student answer word sequence and the standard answer word sequence according to the step S1.1 to respectively obtain student answer word embedded sequences
Figure BDA0003668781920000123
Standard answer word embedding sequence
Figure BDA0003668781920000124
Then embedding the test question words into the sequence T e Respectively embedding the answer words with students
Figure BDA0003668781920000125
Standard answer word embedding sequence
Figure BDA0003668781920000126
Splicing the head and the tail to obtain an answer input sequence A of the student e And a standard answer input sequence S e The calculation process is as follows:
Figure BDA0003668781920000127
Figure BDA0003668781920000128
wherein, ": "denotes the end-to-end concatenation operation of the sequence, n1 is the sentence length of student answers, n2 is the sentence length of standard answers, m1 is the student answer input sequence A e And m1 is n + n1, m2 is the standard answerInput sequence S e And m2 ═ n + n 2;
s2.2 converting the test question type y obtained in the step S1 into the corresponding test question type embedding through table lookup operation
Figure BDA0003668781920000129
Then v is measured t Respectively input with student's answer input sequence A e And a standard answer input sequence S e Each word in the Chinese character string is embedded and connected to obtain student answer perception input A t And standard answer perception input S t The calculation process is as follows:
Figure BDA00036687819200001210
Figure BDA00036687819200001211
wherein the content of the first and second substances,
Figure BDA00036687819200001212
respectively, student's answer input sequence A e And student answer perception input A t Word-embedding vector of the ith column, d t Is the dimension of the embedding of the test question types,
Figure BDA00036687819200001213
respectively, standard answer input sequence S e And standard answer perception input S t The word in column j embeds a vector, "; "represents a concatenation operation of vectors;
s2.3 inputting student answer perception A t And standard answer perception input S t Respectively sending the data to two convolutional neural networks which adopt zero padding and share parameters to obtain shallow semantic representation A of student answers c And the standard answer shallow semantic representation S c The calculation process is as follows:
Figure BDA0003668781920000131
Figure BDA0003668781920000132
wherein sharedPCNN1(·), sharedPCNN2 represent two convolutional neural networks with zero padding and shared parameters, d c The number of convolution kernels;
s2.4 shallow semantic representation A of student answers c And the standard answer shallow semantic representation S c Sending the data into an interactive network consisting of Bi-LSTM to obtain a scoring vector Z of student answers g As follows:
Figure BDA0003668781920000133
where Interactive (. cndot.) represents an Interactive network consisting of Bi-LSTM, d L Dimension of hidden state of Bi-LSTM in interactive network;
s2.5 Scoring vector Z to student' S answer g Performing linear transformation of softmax, performing probability calculation of the grade, and obtaining a final grade, wherein the calculation process is as follows:
x＝M g Z g +b g (18)
Figure BDA0003668781920000134
Figure BDA0003668781920000135
wherein the content of the first and second substances,
Figure BDA0003668781920000136
is a matrix of the score level representation,
Figure BDA0003668781920000137
is an offset vector, d g Is the number of the rating levels to be scored,
Figure BDA0003668781920000138
is represented by Z g Vector of confidence scores of all the rating levels, g being one rating level, x g Represents Z g Confidence score, x, labeled as rating g i Represents Z g Confidence score on the i-th scoring level, Ρ (g | Z) g ) Representing a score vector Z g Prediction probability on a score level G, G being the set of all score levels, G * In order to achieve the final rating of the rating scale,
Figure BDA0003668781920000139
denotes return such that Pp (g | Z) g ) The score rating of the maximum value.
Further, the step S2.4 specifically includes:
s2.4.1 shallow semantic representation A of student answers using two Bi-LSTMs sharing parameters respectively c And the standard answer shallow semantic representation S c Encoding to obtain student answer context expression A r And a standard answer context representation S r The calculation process is as follows:
Figure BDA0003668781920000141
Figure BDA0003668781920000142
wherein ShardedBilSTM 1 (-), ShardedBilSTM 2 (-), represent Bi-LSTM sharing two parameters,
Figure BDA0003668781920000143
respectively represent A c And S c The connection result of the Bi-directional hidden state corresponding to the ith time step in the Bi-LSTM,
Figure BDA0003668781920000144
respectively represent A c And S c From left to right
Figure BDA0003668781920000145
Hidden state at the ith time step in the network,
Figure BDA0003668781920000146
respectively represent A c And S c In a right to left direction
Figure BDA0003668781920000147
Hidden state at ith time step in the network;
s2.4.2 representation of student answer context A through attention mechanism r And a standard answer context representation S r Interact to get mutual attention
Figure BDA0003668781920000148
And
Figure BDA0003668781920000149
the calculation process is as follows;
Figure BDA00036687819200001410
Figure BDA00036687819200001411
Figure BDA00036687819200001412
Figure BDA00036687819200001413
Figure BDA00036687819200001414
Figure BDA00036687819200001415
wherein the content of the first and second substances,
Figure BDA00036687819200001416
is composed of
Figure BDA00036687819200001417
The (c) th element of (a),
Figure BDA00036687819200001418
is composed of
Figure BDA00036687819200001419
The (c) th element of (a),
Figure BDA00036687819200001420
respectively represent pair
Figure BDA00036687819200001421
And
Figure BDA00036687819200001422
the transposition operation is carried out and the operation is carried out,
Figure BDA00036687819200001431
to refer to a logical quantity of "all",
Figure BDA00036687819200001423
respectively representing hidden state tuples
Figure BDA00036687819200001424
And
Figure BDA00036687819200001425
attention weight of (a);
s2.4.3 respectively will be
Figure BDA00036687819200001426
And
Figure BDA00036687819200001428
with student answer context representation A r And a standard answer context representation S r Fusing to obtain student answer fusion expression A f Fused representation S with standard answers f The calculation process is as follows:
Figure BDA00036687819200001429
Figure BDA00036687819200001430
wherein the content of the first and second substances,
Figure BDA0003668781920000151
is A f The (i) th element of (a),
Figure BDA0003668781920000152
is S f The ith element in (1), Fusion (-), represents a Fusion operation, and an element-by-element multiplication operation of the matrix;
s2.4.4 fusion of student answers using two shared parameters Bi-LSTM respectively f Fused representation S with standard answers f Performing enhancement processing to obtain deep semantic representation A of student answers u And standard answer deep semantic representation S u The calculation process is as follows:
Figure BDA0003668781920000153
Figure BDA0003668781920000154
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003668781920000155
respectively represent A f And S f The connection result of the Bi-directional hidden state corresponding to the ith and jth time steps in the Bi-LSTM;
s2.4.5 deep semantic representation A of student answers respectively u And standard answer deep semantic representation S u Carrying out average pooling and maximum pooling, and connecting all pooled vectors to obtain score vector Z of student answer g The calculation process is as follows:
Figure BDA0003668781920000156
Figure BDA0003668781920000157
Figure BDA0003668781920000158
Figure BDA0003668781920000159
Figure BDA00036687819200001510
wherein maxPolling (. cndot.) represents the maximum pooling operation, avePolling (. cndot.) represents the average pooling operation,
Figure BDA00036687819200001511
is A u The maximum pooling vector of (a) is,
Figure BDA00036687819200001512
is A u The average pooling vector of (a) is,
Figure BDA00036687819200001515
is S u The maximum pooling vector of (a) is,
Figure BDA00036687819200001513
is S u The average pooling vector of "; "represents a concatenation operation of vectors.
Further, the step S3 specifically includes:
s3.1, calculating a loss function of the test question classification submodel and a loss function of the subjective question automatic scoring submodel by using the cross entropy loss error respectively, wherein the calculation process is as follows:
Figure BDA00036687819200001514
Figure BDA0003668781920000161
wherein, omega is a training set of the subjective question automatic evaluation task of the joint test question classification and evaluation learning, | omega | represents the size of the training set omega,
Figure BDA0003668781920000162
is the real test question classification of the ith training sample in omega,
Figure BDA0003668781920000163
is the true score level of the ith training sample in omega,
Figure BDA0003668781920000164
is the classification vector of the test question of the ith training sample in omega,
Figure BDA0003668781920000165
is the score vector of student answer of the ith training sample in omega, gamma 1 Is a loss function used during the training of the test question classification submodel, upsilon 2 The method is a loss function used during the sub-model training of automatic scoring of the subjective questions;
s3.2 calculating the automatic review god of the subjective questions of the combined test question classification and the scoring learning by using the following formulaJoint loss function via network model
Figure BDA0003668781920000166
Figure BDA0003668781920000167
Where λ and β are two weight parameters;
s3.3 subjective question automatic review neural network model training objective of joint test question classification and scoring learning is to minimize the joint loss error of equation (40).
Further, the table lookup operation in step S2.2 specifically includes:
Figure BDA0003668781920000168
Figure BDA0003668781920000169
wherein, OneHot (y) * ) Label y for indicating to-be-tested question type * Is converted into a one-hot vector,
Figure BDA00036687819200001610
as test question type y * The one-hot vector of (a) is,
Figure BDA00036687819200001611
embedding matrices for question types, d y Is the number of categories of test question types, d t Is the dimension of the question type embedding.
Examples of applications
1. Example hyper-parameters
TABLE 1 example model hyper-parameter settings
Figure BDA00036687819200001612
Figure BDA0003668781920000171
2. Data set
The data of this example comes from end-of-term examination questions of computer network of the department of the university software engineering specialty of the same year, to obtain 3948 chinese subjective question questions, each question is provided with 1 standard answer and 4 student answers, and finally, a question classification data set containing 3948 chinese subjective question questions and an automatic scoring data set of 15792 chinese subjective questions with answers are obtained, as shown in tables 2 and 3:
TABLE 2 question classification dataset
Figure BDA0003668781920000172
TABLE 3 automatic scoring dataset partitioning
Figure BDA0003668781920000173
3. Examples comparative results
Table 4 shows the classification result comparison results of the test question types of the various models, and table 5 shows the classification comparison results of the student answer score intervals of the various models:
TABLE 4 comparison of the results of the classification of test question types for various models
Model (model) Accuracy Recall Precision F1 value
Test question classification model for joint scoring 96.03 96.1 95.60 95.85
Independent test question classification model 94.56 94.21 94.56 94.38
TABLE 5 results of classification comparison of student answer score intervals for various models
Figure BDA0003668781920000174
Figure BDA0003668781920000181
The experimental results in tables 4 and 5 show that the automatic deep learning method for subjective question review by combined test question classification and scoring learning provided by the invention is obviously superior to the independent review model and the independent test question classification model, and is obviously superior to the review model without the CNN layer and the review model without the Bi-LSTM interaction layer, which fully proves that the method of the invention is feasible and excellent.
4. Examples of the invention
Table 6 gives examples of student answer scores for various test question types:
TABLE 6 student answer Scoring example for various test question types
Figure BDA0003668781920000182

Claims (5)

1. A subjective question automatic evaluation deep learning method combining test question classification and evaluation learning is characterized by comprising the following steps:
s1, classifying the test questions of the subjective questions to be evaluated through a CNN test question classification submodel with residual connection to obtain the types of the test questions of the subjective questions to be evaluated;
s2, embedding and connecting test question types of subjective questions to be evaluated with each character of a student answer input sequence and a standard answer input sequence to obtain student answer perception input and standard answer perception input of test question type perception, and then sending the student answer perception input and the standard answer perception input into a subjective question test paper scoring sub-model consisting of CNN and Bi-LSTM to score, so as to obtain scoring grades of the student answers;
s3, training a subjective question automatic evaluation neural network model for combined test question classification and evaluation learning through the loss functions of the combined calculation step S1 and the step S2;
s4, automatically evaluating the neural network model by using the formed subjective questions trained in the step S3, and evaluating any student answers;
CNN is an abbreviation for the English conditional Neural Networks, representing a Convolutional Neural network, and is described by the documents "Lawrence S, Giles CL, Tsio AC et al. face repetition: A conditional Neural Networks, IEEE Transactions on 1997; 8(1) 98-113';
the Bi-LSTM is an abbreviation of English Directional Long Short-Term Memory and represents a two-way Long-Short Term Memory Neural network, wherein the LSTM Neural network is proposed by the documents "Sepp Hochreiter and Juregen Schmidhuber.1997 Long Short-Term Memory, 9(8): 1735-;
the subjective question types comprise definition subjective question test questions, sequence subjective question test questions and general subjective question test questions, wherein the definition subjective question test questions refer to subjective questions requiring students to explain definition of a certain concept, the sequence subjective question test questions refer to subjective questions with answers comprising a plurality of key points given in sequence, and the general subjective question test questions refer to other subjective questions except the definition and the sequence.
2. The method for automatic deep learning of subjective questions by combined test question classification and score learning according to claim 1, wherein:
the step S1 specifically includes:
s1.1 character sequence of test question to be classified
Figure FDA0003668781910000011
Word embedding sequence T converted into test question to be classified through table look-up operation e The calculation process is as follows:
Figure FDA0003668781910000012
Figure FDA0003668781910000021
wherein n represents the sentence length of the test question to be classified, onehot (T) represents the conversion of the test question word sequence T into a unique heat matrix, T o Is a one-hot matrix of the test word sequence T, | V | represents the size of a dictionary V, which is a dictionary established by a model and containing all words,
Figure FDA0003668781910000022
a matrix is embedded for the pre-trained words corresponding to dictionary V,
Figure FDA0003668781910000023
representing a word embedding vector corresponding to the ith word in the test question word sequence T, and h representing the dimension of the word embedding vector;
S1.2 embedding the words of the test questions to be classified into the sequence T e Inputting the data into a convolutional neural network adopting zero filling to obtain shallow semantic representation T of the test question to be classified c The calculation process is as follows:
Figure FDA0003668781910000024
Figure FDA0003668781910000025
wherein PCNN (. cndot.) represents a convolutional neural network with zero padding, (T) c ) ij Represents T c I row and j column of (c), k is the convolution kernel size, d c Is the number of the convolution kernels and is,
Figure FDA0003668781910000026
word-embedded sequence T representing test questions to be classified e As a result of the zero-padding being performed,
Figure FDA0003668781910000027
to represent
Figure FDA0003668781910000028
The j (th) column to the j + (k-1) column of the matrix,
Figure FDA0003668781910000029
the weight of the ith convolution kernel is represented,
Figure FDA00036687819100000210
represents the offset of the ith convolution kernel,
Figure FDA00036687819100000211
representing a convolution operation;
s1.3 shallow semantic representation T of test questions to be classified c Is sent into a full-connected twoThe feedforward neural network formed by the connected layers is enhanced to obtain the enhanced semantic representation of the test questions to be classified
Figure FDA00036687819100000212
The calculation process is as follows:
Figure FDA00036687819100000213
wherein, W 1 f ,
Figure FDA00036687819100000214
Respectively the weight matrices of the two fully-connected layers,
Figure FDA00036687819100000215
bias vectors of two fully-connected layers respectively, ReLU is an abbreviation of English 'Rectified Linear Unit', and ReLU (·) represents an activation function adopting modified Linear Unit operation;
s1.4, expressing T by shallow semantic of test questions to be classified c With its enhanced semantic representation T f Residual error connection is carried out to obtain deep semantic representation T of the test question to be classified u The calculation process is as follows:
Figure FDA00036687819100000216
wherein LayerNorm (·) denotes layer normalization;
s1.5 deep semantic representation T of test questions to be classified u Performing maximum pooling to obtain a classification vector Z of the test questions to be classified c The calculation process is as follows:
Figure FDA00036687819100000217
wherein maxPooling (·) represents a max pooling operation;
s1.6 Classification vector Z of test questions to be classified c Executing linear transformation of softmax, performing probability calculation of the test question types, and obtaining the final test question types, wherein the calculation process is as follows:
o＝M c Z c +b o (8)
Figure FDA0003668781910000031
Figure FDA0003668781910000032
wherein the content of the first and second substances,
Figure FDA0003668781910000033
is a matrix of the test question type representation,
Figure FDA0003668781910000034
is an offset vector, d y Is the number of the categories of the test question types,
Figure FDA0003668781910000035
represents Z c Confidence score vectors over all test question types, y being one test question type, o y Represents Z c Confidence score, o, labeled test question type y i Represents Z c Confidence score, Ρ (yz | Z) on the ith test question type c ) Classification vector Z representing test questions to be classified c The prediction probability over the test question type Y, which is the set of all test question types,
Figure FDA0003668781910000036
the representation returns such that Pp (y | Z) c ) Type of test question, y, as maximum * For the finally classified test question type of the test questions to be classified, exp (-) represents an exponential function with a natural constant e as the base.
3. The method for automatically deep learning by reviewing subjective questions combined with test question classification and score learning according to claim 1, wherein:
the step S2 specifically includes:
s2.1, performing table look-up operation on the student answer word sequence and the standard answer word sequence according to the step S1.1 to respectively obtain student answer word embedded sequences
Figure FDA0003668781910000037
Standard answer word embedding sequence
Figure FDA0003668781910000038
Then embedding the test question words into the sequence T e Respectively embedding the answer words with students
Figure FDA0003668781910000039
Standard answer word embedding sequence
Figure FDA00036687819100000310
Splicing the head and the tail to obtain an answer input sequence A of the student e And a standard answer input sequence S e The calculation process is as follows:
Figure FDA00036687819100000311
Figure FDA00036687819100000312
wherein, ": "denotes the end-to-end concatenation operation of the sequence, n1 is the sentence length of student answers, n2 is the sentence length of standard answers, m1 is the student answer input sequence A e M1 ═ n + n1, m2 is the standard answer input sequence S e And m2 ═ n + n 2;
s2.2, the test question types y obtained in the step S1 are passedThe table look-up operation is converted into the corresponding test question type to be embedded
Figure FDA0003668781910000048
Then v is measured t Respectively input with student's answer input sequence A e And a standard answer input sequence S e Each character in the Chinese character string is embedded and connected to obtain student answer perception input A t And standard answer perception input S t The calculation process is as follows:
Figure FDA0003668781910000041
Figure FDA0003668781910000042
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003668781910000043
respectively, student's answer input sequence A e And student answer perception input A t Word-embedding vector of the ith column, d t Is the dimension of the embedding of the test question types,
Figure FDA0003668781910000044
respectively, standard answer input sequence S e And standard answer perception input S t The word in column j embeds a vector, "; "represents a concatenation operation of vectors;
s2.3 inputting student answer perception A t And standard answer perception input S t Respectively sending the data to two convolutional neural networks which adopt zero padding and share parameters to obtain shallow semantic representation A of student answers c And the standard answer shallow semantic representation S c The calculation process is as follows:
Figure FDA0003668781910000045
Figure FDA0003668781910000046
wherein sharedPCNN1(·), sharedPCNN2 represent two convolutional neural networks with zero padding and shared parameters, d c The number of convolution kernels;
s2.4 shallow semantic representation A of student answers c And the standard answer shallow semantic representation S c Sending the data into an interactive network consisting of Bi-LSTM to obtain a scoring vector Z of student answers g As follows:
Figure FDA0003668781910000047
wherein Interactive (-) represents an Interactive network composed of Bi-LSTM, d L Dimension of hidden state of Bi-LSTM in interactive network;
s2.5 Scoring vector Z to student' S answer g Performing linear transformation of softmax, performing probability calculation of the grade, and obtaining a final grade, wherein the calculation process is as follows:
x＝M g Z g +b g (18)
Figure FDA0003668781910000051
Figure FDA0003668781910000052
wherein the content of the first and second substances,
Figure FDA0003668781910000053
is a matrix of the score level representation,
Figure FDA0003668781910000054
is an offset vector, d g Is the number of the rating levels to be scored,
Figure FDA0003668781910000055
is represented by Z g Vector of confidence scores of all the rating levels, g being one rating level, x g Represents Z g Confidence score, x, labeled as rating g i Represents Z g Confidence score on the i-th scoring level, Ρ (g | Z) g ) Representing a score vector Z g Prediction probability on a score level G, G being the set of all score levels, G * In order to achieve the final rating of the rating scale,
Figure FDA0003668781910000056
denotes return such that Pp (g | Z) g ) The score rating of the maximum value.
4. The automatic deep learning method for subjective questions assessment combined with test question classification and assessment learning according to claim 3, wherein:
the step S2.4 specifically includes:
s2.4.1 shallow semantic representation A of student answers using two Bi-LSTMs sharing parameters respectively c And the standard answer shallow semantic representation S c Encoding to obtain student answer context expression A r And a standard answer context representation S r The calculation process is as follows:
Figure FDA0003668781910000057
Figure FDA0003668781910000058
wherein ShardedBilSTM 1 (-), ShardedBilSTM 2 (-), represent Bi-LSTM sharing two parameters,
Figure FDA0003668781910000059
Figure FDA00036687819100000510
respectively represent A c And S c The connection result of the Bi-directional hidden state corresponding to the ith time step in the Bi-LSTM;
s2.4.2 context representation of student answers by attention mechanism A r And a standard answer context representation S r Interact to get mutual attention
Figure FDA00036687819100000511
And
Figure FDA00036687819100000512
the calculation process is as follows;
Figure FDA00036687819100000513
Figure FDA00036687819100000514
Figure FDA00036687819100000515
Figure FDA00036687819100000516
Figure FDA0003668781910000061
Figure FDA0003668781910000062
wherein the content of the first and second substances,
Figure FDA0003668781910000063
is composed of
Figure FDA0003668781910000064
The (c) th element of (a),
Figure FDA0003668781910000065
is composed of
Figure FDA0003668781910000066
The (c) th element of (a),
Figure FDA0003668781910000067
respectively represent pair
Figure FDA0003668781910000068
And
Figure FDA0003668781910000069
the transposition operation is carried out and the operation is carried out,
Figure FDA00036687819100000610
to refer to a logical quantity of "all",
Figure FDA00036687819100000611
respectively representing hidden state tuples
Figure FDA00036687819100000612
And
Figure FDA00036687819100000613
attention weight of (1);
s2.4.3 respectively will be
Figure FDA00036687819100000614
And
Figure FDA00036687819100000615
with student answer context representation A r And a standard answer context representation S r Fusing to obtain student answer fusion expression A f Fused representation S with standard answers f The calculation process is as follows:
Figure FDA00036687819100000616
Figure FDA00036687819100000617
wherein the content of the first and second substances,
Figure FDA00036687819100000618
is A f The (c) th element of (a),
Figure FDA00036687819100000619
is S f The ith element in (1), Fusion (-), represents a Fusion operation, and an element-by-element multiplication operation of the matrix;
s2.4.4 fusion of student answers using two Bi-LSTMs sharing parameters respectively f Fused representation S with standard answers f Performing enhancement processing to obtain deep semantic representation A of student answers u And standard answer deep semantic representation S u The calculation process is as follows:
Figure FDA00036687819100000620
Figure FDA00036687819100000621
wherein, the first and the second end of the pipe are connected with each other,
Figure FDA00036687819100000622
respectively represent A f And S f The connection result of the Bi-directional hidden state corresponding to the ith and jth time steps in the Bi-LSTM;
s2.4.5 deep semantic representation A of student answers respectively u And standard answer deep semantic representation S u Carrying out average pooling and maximum pooling, and connecting all pooled vectors to obtain score vector Z of student answer g The calculation process is as follows:
Figure FDA00036687819100000623
Figure FDA00036687819100000624
Figure FDA00036687819100000625
Figure FDA00036687819100000626
Figure FDA00036687819100000627
wherein maxPooling (. cndot.) represents the maximum pooling operation, avePooling (. cndot.) represents the average pooling operation,
Figure FDA0003668781910000071
is A u The maximum pooling vector of (a) is,
Figure FDA0003668781910000072
is A u The average pooling vector of (a) is,
Figure FDA0003668781910000073
is S u The maximum pooling vector of (a) is,
Figure FDA0003668781910000074
is S u The average pooling vector of "; "represents a concatenation operation of vectors.
5. The method for automatic deep learning of subjective questions by combined test question classification and score learning according to claim 1, wherein:
the step S3 specifically includes:
s3.1, calculating a loss function of the test question classification submodel and a loss function of the subjective question automatic scoring submodel by using the cross entropy loss error respectively, wherein the calculation process is as follows:
Figure FDA0003668781910000075
Figure FDA0003668781910000076
wherein, omega is a training set of the subjective question automatic evaluation task of the joint test question classification and evaluation learning, | omega | represents the size of the training set omega,
Figure FDA0003668781910000077
is the real test question classification of the ith training sample in omega,
Figure FDA0003668781910000078
is the true score level of the ith training sample in omega,
Figure FDA0003668781910000079
is the ith training sample in ΩThe classification vector of the test question of the present invention,
Figure FDA00036687819100000710
the score vector of student answer, y, of the ith training sample in Ω 1 Is a loss function used in the training of the test question classification submodel, gamma 2 Is a loss function used when carrying out the automatic scoring submodel training of the subjective questions;
s3.2 calculating the joint loss function of the automatic evaluation neural network model of the subjective questions of the joint test question classification and evaluation learning by using the following formula
Figure FDA00036687819100000711
Figure FDA00036687819100000712
Wherein λ and β are two weight parameters;
s3.3 subjective question automatic review neural network model training objective of joint test question classification and scoring learning is to minimize the joint loss error of equation (40).
CN202210597773.7A 2022-05-30 2022-05-30 Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning Pending CN114969260A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210597773.7A CN114969260A (en) 2022-05-30 2022-05-30 Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210597773.7A CN114969260A (en) 2022-05-30 2022-05-30 Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning

Publications (1)

Publication Number Publication Date
CN114969260A true CN114969260A (en) 2022-08-30

Family

ID=82957945

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210597773.7A Pending CN114969260A (en) 2022-05-30 2022-05-30 Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning

Country Status (1)

Country Link
CN (1) CN114969260A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827879A (en) * 2023-02-15 2023-03-21 山东山大鸥玛软件股份有限公司 Low-resource text intelligent review method and device based on sample enhancement and self-training
CN117034954A (en) * 2023-10-09 2023-11-10 华南师范大学 Text scoring method, device, equipment and storage medium
CN117252739A (en) * 2023-11-17 2023-12-19 山东山大鸥玛软件股份有限公司 Method, system, electronic equipment and storage medium for evaluating paper

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115827879A (en) * 2023-02-15 2023-03-21 山东山大鸥玛软件股份有限公司 Low-resource text intelligent review method and device based on sample enhancement and self-training
CN117034954A (en) * 2023-10-09 2023-11-10 华南师范大学 Text scoring method, device, equipment and storage medium
CN117034954B (en) * 2023-10-09 2024-02-06 华南师范大学 Text scoring method, device, equipment and storage medium
CN117252739A (en) * 2023-11-17 2023-12-19 山东山大鸥玛软件股份有限公司 Method, system, electronic equipment and storage medium for evaluating paper
CN117252739B (en) * 2023-11-17 2024-03-12 山东山大鸥玛软件股份有限公司 Method, system, electronic equipment and storage medium for evaluating paper

Similar Documents

Publication Publication Date Title
CN111554268B (en) Language identification method based on language model, text classification method and device
CN110245229B (en) Deep learning theme emotion classification method based on data enhancement
CN108549658B (en) Deep learning video question-answering method and system based on attention mechanism on syntax analysis tree
CN114969260A (en) Automatic subjective question evaluation and reading deep learning method combining test question classification and evaluation learning
Uto A review of deep-neural automated essay scoring models
CN111753098A (en) Teaching method and system based on cross-media dynamic knowledge graph
CN113656570A (en) Visual question answering method and device based on deep learning model, medium and equipment
CN110516070B (en) Chinese question classification method based on text error correction and neural network
CN111985239A (en) Entity identification method and device, electronic equipment and storage medium
CN109101490B (en) Factual implicit emotion recognition method and system based on fusion feature representation
CN113204633B (en) Semantic matching distillation method and device
CN113569001A (en) Text processing method and device, computer equipment and computer readable storage medium
CN112131883A (en) Language model training method and device, computer equipment and storage medium
CN112232053A (en) Text similarity calculation system, method and storage medium based on multi-keyword pair matching
CN111897954A (en) User comment aspect mining system, method and storage medium
CN111581364B (en) Chinese intelligent question-answer short text similarity calculation method oriented to medical field
CN113704392A (en) Method, device and equipment for extracting entity relationship in text and storage medium
CN113342958A (en) Question-answer matching method, text matching model training method and related equipment
CN110874392B (en) Text network information fusion embedding method based on depth bidirectional attention mechanism
CN116186237A (en) Entity relationship joint extraction method based on event cause and effect inference
CN112131345A (en) Text quality identification method, device, equipment and storage medium
CN113934835B (en) Retrieval type reply dialogue method and system combining keywords and semantic understanding representation
CN115935991A (en) Multitask model generation method and device, computer equipment and storage medium
CN116975350A (en) Image-text retrieval method, device, equipment and storage medium
CN115017879A (en) Text comparison method, computer device and computer storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination