CN107590127B - Automatic marking method and system for question bank knowledge points - Google Patents
Automatic marking method and system for question bank knowledge points Download PDFInfo
- Publication number
- CN107590127B CN107590127B CN201710859784.7A CN201710859784A CN107590127B CN 107590127 B CN107590127 B CN 107590127B CN 201710859784 A CN201710859784 A CN 201710859784A CN 107590127 B CN107590127 B CN 107590127B
- Authority
- CN
- China
- Prior art keywords
- word
- vector
- target
- topic
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
the invention provides an automatic marking method and system for question bank knowledge points, wherein the method comprises the following steps: preprocessing each topic in a topic library, acquiring a target topic with an answer vocabulary filled in a null symbol, and acquiring a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title; and determining knowledge points for topic investigation according to the feature vectors of the target topics and preset knowledge points. The invention predicts the knowledge points according to the word vector and word sequence information of each word in the title, fully utilizes the information of the text and can achieve better labeling effect.
Description
Technical Field
The invention relates to the technical field of computers, in particular to an automatic labeling method and system for question bank knowledge points.
background
With the development of the internet industry, electronic homework and quizzes have become popular methods for evaluating students. In order to improve the learning efficiency and reduce the burden of students, personalized learning strategies are customized according to the abilities and the defects of the students, and suitable questions are pushed to carry out self-adaptive testing, so that one of the goals pursued by internet education is achieved. The question bank which is structured and rich in semantic information is the premise of the self-adaptive test. To construct a structured question bank, it is a desirable and feasible way to provide metadata (metadata) for the question bank using a knowledge-graph consisting of curriculum knowledge points or concepts and associations between them. At present, a knowledge point label is marked on a question to associate a question bank with a knowledge map, and most of the questions adopt a manual labeling mode, which greatly consumes manpower and financial resources and puts higher requirements on the professional level of a label maker. Since annotators may analyze the problem from different angles, consistency and accuracy of the annotation is more difficult to guarantee. Therefore, the method has very important significance in marking the question bank by adopting a machine.
most of existing text labeling methods based on machine learning characterize texts based on bag-of-words models (bag-of-words), and context information of words is ignored. Most computer tests adopt a mode of objective questions, which are usually expressed as short texts, and available information is small. Especially for the subjects of language class courses, the sequence information of the vocabulary is especially critical to reflect knowledge points. Therefore, the traditional labeling method cannot show good effect on the labeling of the question bank. In addition, some semantic tagging (semantic tagging) methods use domain ontology to tag objects. These labeling methods often rely on good conceptual (label) descriptions. However, for the question bank, not all knowledge points of the course have rich concept descriptions. Moreover, when the topic content and the concept description have low correlation, that is, there are not too many co-occurring words or near-meaning words between them, the labeling effect is greatly reduced. Therefore, the project provides an effective way for carrying out knowledge point labeling according to the characteristics of the commonly used objective questions of the electronic test.
Disclosure of Invention
the present invention provides a method and system for automatically labeling problem base knowledge points that overcomes or at least partially solves the above-mentioned problems.
In a first aspect, the present invention provides an automatic labeling method for question bank knowledge points, including:
preprocessing each topic in a topic library, acquiring a target topic with an answer vocabulary filled in a null symbol, and acquiring a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
Calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title;
and determining knowledge points for topic investigation according to the feature vectors of the target topics and preset knowledge points.
preferably, the preprocessing is performed on each topic in the topic library to obtain a target topic with an empty symbol, and the method includes:
Preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
Calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
and calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
Preferably, the adjustment vector is a scaling factor of the attention weight of each word;
then, calculating the feature vector of the target topic according to the feature vector of each word, the attention weight of each word and a preset adjustment vector, including:
Calculating the feature vector of the target object by formula (I) according to the feature vector of each word, the attention weight of each word and a preset adjustment vector
r ═ s1 ═ h1+ s2 ═ h2+ … + sn-1 × -hn-1 + sn × -hn formula (one)
wherein ai is pi × ws, r is a feature vector of the target topic, pi is an attention weight of the ith word, ws is a scaling factor of the attention weight of each word, h1., hn is a feature vector of the first word to the nth word respectively, and n is the total number of words of the target topic.
preferably, the adjustment vector includes three local adjustment factors, which are a first local adjustment factor, a second local adjustment factor and a third local adjustment factor, respectively, and the three local adjustment factors are all non-negative numbers and sum to 1;
then, according to the feature vector of each word, the attention weight of each word and a preset adjustment vector, calculating the feature vector of the target topic by formula (I)
r ═ s1 ═ h1+ s2 ═ h2+ … + sn-1 × -hn-1 + sn × -hn formula (one)
Wherein r is a feature vector of the target topic, p1 and p2 are attention weights of a first word and a second word respectively, pi-1, pi and pi +1 are attention weights of an i-1 th word, an i-th word and an i +1 th word respectively, pn-1 and pn are attention weights of an n-1 th word and an n-th word respectively, wl1, wl2 and wl3 are first local adjustment factors, second local adjustment factors and third local adjustment factors respectively, h1., hn is a feature vector of the first word to the n-th word respectively, and n is the total number of words of the target topic.
Preferably, the preprocessing is performed on each topic in the topic library to obtain a target topic with an empty symbol, and the method includes:
Preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
Calculating a forward feature vector of each word and a backward feature vector of each word in the target topic according to the word vector of each word and the word sequence information of the target topic and a first preset strategy;
Calculating the feature vector of the target title according to the forward feature vector of each word and the backward feature vector of each word; the first preset strategy is as follows:
calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of a first word according to the word vector, a preset forward initial characteristic vector and a preset forward initial cell state value of the first word in the target topic;
calculating an input value, a forgetting value, an output value, a cell state value and a backward characteristic vector of the last word according to the word vector, a preset backward initial characteristic vector and a preset backward initial cell state value of the last word in the target topic;
sequentially and iteratively calculating the forward characteristic vector of each word from the first word to the current word according to the input value, the forgetting value, the output value, the cell state value, the forward characteristic vector of the first word and the word vector of each word between the first word and the current word, and obtaining the forward characteristic vector of each current word;
and sequentially carrying out iterative computation until the current word according to the sequence from the last word to the current word and obtaining the backward characteristic vector of each current word according to the input value, the forgetting value, the output value, the cell state value, the backward characteristic vector of the last word and the word vector of each word between the last word and the current word.
Preferably, the preprocessing is performed on each topic in the topic library to obtain a target topic with an empty symbol, and the method includes:
Preprocessing each question in the question bank, and acquiring a target question with repeatedly filled with multiple answer words at each preset null symbol;
Then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
Calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of the first word according to the word vector of the first word, a preset forward initial characteristic vector and a preset forward initial cell state value;
sequentially and iteratively calculating until the last word according to the input value, the forgetting value, the output value, the cell state value and the feature vector of the first word and the sequence from the first word to the last word, and obtaining the forward feature vector of the last word;
calculating the backward characteristic vector of the last word according to the word vector of the last word in the target topic, a preset backward initial characteristic vector and a preset backward initial cell state value; and calculating the feature vector of the target topic according to the forward feature vector of the last word and the backward feature vector of the last word.
preferably, determining the knowledge points of topic investigation according to the feature vector of the target topic and preset knowledge points comprises:
Calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
alternatively, the first and second electrodes may be,
sequencing the investigation probabilities from large to small;
And confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
in a second aspect, the present invention further provides an automatic labeling system for question bank knowledge points, including:
The first acquisition unit is used for preprocessing each question in the question bank and acquiring a target question with an empty symbol filled with an answer vocabulary;
the second obtaining unit is used for obtaining a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
The calculation unit is used for calculating the feature vector of the target title according to a preset strategy according to the word vector of each word and the target word sequence information of the target title;
And the determining unit is used for determining the knowledge points of topic investigation according to the feature vectors of the target topics and the preset knowledge points.
Preferably, the first obtaining unit is further configured to:
preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, the computing unit is further configured to:
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
and calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
Preferably, the determining unit is further configured to:
calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
Confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
Alternatively, the first and second electrodes may be,
Sequencing the investigation probabilities from large to small;
and confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
According to the technical scheme, the knowledge points are predicted according to the word vector and word sequence information of each word in the title, the information of the text is fully utilized, and a good labeling effect can be achieved.
Drawings
FIG. 1 is a flowchart illustrating a method for automatically labeling knowledge points in an item library according to an embodiment of the present invention;
FIG. 2 is an exemplary diagram of a single choice question for an English course;
FIG. 3 is a schematic diagram of a network structure of a location-based attention model;
FIG. 4a is a diagram illustrating a first way of adjusting attention weights;
FIG. 4b is a diagram of a second way of adjusting attention weights;
FIG. 5 is a schematic diagram of a network architecture based on a keyword model;
FIG. 6 is a diagram illustrating an example of topic label prediction.
FIG. 7 is a schematic block diagram of an automatic question bank knowledge point annotation system according to an embodiment of the present invention;
Detailed Description
the following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
for objective questions, it usually exhibits the following characteristics: (1) one problem is often presented in short text and therefore less information is available. (2) In a question, several nulls are usually required to be filled in, and the position where the nulls appear can be treated as a kind of prompt message. (3) An objective question is often composed of a stem, options and answers. Whether they are integrated together for processing or separately is a matter of consideration. Meanwhile, the objective question is intuitively felt that the answer of the question can reflect the knowledge point of the question investigation. In addition, the vocabulary around the answer may also play an important role in reflecting the knowledge points of topic investigation. Thus, based on these features of objective topics, a location-based attention model (position-based attention model) and a keyword-based model (keywords-based model) are proposed for topic tagging. As described in detail below.
Fig. 1 is a flowchart of an automatic question bank knowledge point labeling method according to an embodiment of the present invention.
The method for automatically labeling question bank knowledge points as shown in fig. 1 comprises the following steps:
s101, preprocessing each topic in a topic library, acquiring a target topic with an answer vocabulary filled in a position with a null symbol, and acquiring a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
the nulls may be underlined, parenthesized, etc.
s102, calculating the feature vector of the target title according to a preset strategy according to the word vector of each word and the target word sequence information of the target title;
s103, determining knowledge points for topic investigation according to the feature vectors of the target topics and preset knowledge points.
the invention predicts the knowledge points according to the word vector and word sequence information of each word in the title, fully utilizes the information of the text and can achieve better labeling effect.
as a preferred embodiment, the preprocessing of each topic in the topic library in step S101 to obtain a target topic with an empty symbol for filling an answer vocabulary includes:
Preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, the step S102 includes:
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
And calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
as a preferred embodiment, the step S103 includes:
Calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
alternatively, the first and second electrodes may be,
sequencing the investigation probabilities from large to small;
And confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
as a preferred embodiment, the adjustment vector is a scaling factor of the attention weight of each word;
then, calculating the feature vector of the target topic according to the feature vector of each word, the attention weight of each word and a preset adjustment vector, including:
calculating the feature vector of the target object through formula 1 according to the feature vector of each word, the attention weight of each word and a preset adjustment vector
r-s 1-h 1+ s 2-h 2+ … + sn-1-hn-1 + sn-hn formula 1
wherein ai is pi × ws, r is a feature vector of the target topic, pi is an attention weight of the ith word, ws is a scaling factor of the attention weight of each word, h1., hn is a feature vector of the first word to the nth word respectively, and n is the total number of words of the target topic.
or the scaling factor comprises three local adjustment factors, namely a first local adjustment factor, a second local adjustment factor and a third local adjustment factor, wherein the three local adjustment factors are all non-negative numbers and the sum is 1;
Then, the feature vector of the target object is calculated according to the feature vector of each word, the attention weight of each word and a preset adjustment vector by formula 1
r-s 1-h 1+ s 2-h 2+ … + sn-1-hn-1 + sn-hn formula 1
wherein r is a feature vector of the target topic, p1 and p2 are attention weights of a first word and a second word respectively, pi-1, pi and pi +1 are attention weights of an i-1 th word, an i-th word and an i +1 th word respectively, pn-1 and pn are attention weights of an n-1 th word and an n-th word respectively, wl1, wl2 and wl3 are first local adjustment factors, second local adjustment factors and third local adjustment factors respectively, h1., hn is a feature vector of the first word to the n-th word respectively, and n is the total number of words of the target topic.
as a preferred embodiment, the preprocessing of each topic in the topic library in step S101 to obtain a target topic with an empty symbol for filling an answer vocabulary includes:
preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, the step S102 includes:
Calculating a forward feature vector of each word and a backward feature vector of each word in the target topic according to the word vector of each word and the word sequence information of the target topic and a first preset strategy;
calculating the feature vector of the target title according to the forward feature vector of each word and the backward feature vector of each word; the first preset strategy is as follows:
Calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of a first word according to the word vector, a preset forward initial characteristic vector and a preset forward initial cell state value of the first word in the target topic;
calculating an input value, a forgetting value, an output value, a cell state value and a backward characteristic vector of the last word according to the word vector, a preset backward initial characteristic vector and a preset backward initial cell state value of the last word in the target topic;
Sequentially and iteratively calculating until the current word according to the input value, the forgetting value, the output value, the cell state value and the forward characteristic vector of the first word and the sequence from the first word to the current word, and acquiring the forward characteristic vector of each current word;
and sequentially and iteratively calculating until the current word according to the input value, the forgetting value, the output value, the cell state value and the feature vector of the last word and the sequence from the last word to the current word, and acquiring the backward feature vector of each current word.
In the embodiment, context representation based on the title fully utilizes context information of the short text, and a better labeling effect can be obtained.
as a preferred embodiment, preprocessing each topic in the topic library to obtain a target topic with an empty symbol filled with an answer vocabulary includes:
preprocessing each question in the question bank, and acquiring a target question with repeatedly filled with multiple answer words at each preset null symbol;
Then, the step S102 includes:
calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of the first word according to the word vector of the first word, a preset forward initial characteristic vector and a preset forward initial cell state value;
Sequentially and iteratively calculating until the last word according to the input value, the forgetting value, the output value, the cell state value and the feature vector of the first word and the sequence from the first word to the last word, and obtaining the forward feature vector of the last word;
calculating the backward characteristic vector of the last word according to the word vector of the last word in the target topic, a preset backward initial characteristic vector and a preset backward initial cell state value; and calculating the feature vector of the target topic according to the forward feature vector of the last word and the backward feature vector of the last word.
In training the model of the invention, the adjustment vector is optimized by the following steps:
firstly, preprocessing each topic in a topic library, acquiring a target topic with an answer vocabulary filled in a position with a null symbol, and acquiring a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
judging whether each word in the title is the word filled in the empty symbol, and endowing each word in the title filled with the answer vocabulary with an initial attention weight according to the judgment result;
calculating a first feature vector of the target object according to the feature vector of each word, the initial attention weight of each word and a preset initial adjustment vector;
Calculating a first investigation probability of each preset knowledge point by the question according to the first feature vector of the target title and the preset knowledge points;
calculating a first cross entropy according to the first investigation probability and a predicted actual probability (knowledge points actually investigated by the questions are known in advance, and the actual probability of the questions to the knowledge points is calculated according to the knowledge points), and calculating a first adjustment vector according to the first cross entropy and the initial adjustment vector;
Calculating a second feature vector of the target title according to the feature vector of each word, the attention weight of each word and the first adjusting vector;
Calculating a second investigation probability of the question to each preset knowledge point according to the second feature vector of the target title and the preset knowledge points;
calculating a second cross entropy according to the second investigation probability and the predicted actual probability;
Judging whether the first adjustment vector is continuously adjusted or not according to a second cross entropy and the first cross entropy;
if not, the first adjusting vector is determined to be the optimal direction adjusting vector.
judging whether to continuously adjust the first adjustment vector according to a second cross entropy and the first cross entropy, wherein the judging comprises the following steps:
Calculating the difference of the first cross entropy minus the second cross entropy;
and if the difference is larger than a preset value, continuing to adjust the first adjusting vector.
wherein calculating a first adjustment vector based on the first cross entropy and the initial adjustment vector comprises:
calculating a first adjustment vector by the following formula according to the first cross entropy and the initial adjustment vector
wherein ws1 is the first adjustment vector, ws0 is the initial adjustment vector, lr is the learning rate, which is a constant greater than 0, and L is the first cross entropy.
The present invention will be described in detail below with reference to specific examples.
1. question bank labeling method of attention model based on position
The idea of the knowledge point of topic investigation can be reflected more based on the answer. The present embodiment uses an attention mechanism to give more attention weight to those words belonging to the answer, so that the final text features contain more of their information. This model is called an attention model. And which positions belong to answer vocabulary and which positions belong to common vocabulary can be obtained by using the prior knowledge of the objective question setting empty positions (which can be obtained by adopting the existing method).
for an english word question as shown in fig. 2, it is processed in the deep neural network as shown in fig. 3 according to the following steps, and finally the labels (knowledge points) are predicted:
(1) Pretreatment: as previously mentioned, a radio topic is typically composed of three parts, and answers are filled into the corresponding spaces in an integrated manner, such as the top left topic input part of FIG. 3. Firstly, filtering out the ambiguous punctuation in the title, and converting the words into lower case. Then, the subject is cut into words, and each word is converted into a one-hot vector x. For example, the word "can" is located at the third position of the dictionary formed by all words in the question bank, and the one-hot vector corresponding to the word "can" is [0,0,1,0 …,0 ]. For the entire channel title, it will be represented as a matrix [ x1, x2, …, xn-1, xn ].
the above-mentioned attention information is generated according to the position of the answer. If the word in one location corresponds to an answer, it will be given more attention, such as "the" and "swing" in FIG. 3.
the initial attention weight assignment for each location is as follows.
then, the location information of a title can be represented as [ p1, p2, …, pn-1, pn ]. The position information [ p1, p2, …, pn-1, pn ] and the topic matrix [ x1, x2, …, xn-1, xn ] are used together as input to the model, and they are in one-to-one correspondence.
(2) word embedding: and performing word Embedding operation on the Embedding Layer, and converting the words represented by one-hot into a word vector containing semantic information by using a word Embedding matrix E.
e＝x×E
each row of the word embedding matrix E corresponds to a word vector of a word. E may be initialized by the probability distribution or by the pre-trained word vector (i.e., its initial value is a fixed value). After the word embedding operation, the title representation is converted from [ x1, x2, …, xn-1, xn ] to [ e1, e2, …, en-1, en ].
(3) topic characterization: to capture the context information, a BilSTM network is used to obtain a representation of the topic containing the context information. Given a topic word vector input [ e1, e2, …, en-1, en ], the LSTM iteratively computes the corresponding hidden state for each time step, i.e., each vocabulary. When processing the t-th time step, the LSTM calculates its corresponding hidden state ht according to the following steps:
i＝sigmoid(We+Wh+Wc+b)
f＝sigmoid(We+Wh+Wc+b)
o＝sigmoid(We+Wh+Wc+b)
c＝f·c+i·tanh(We+Wh+b)
h＝o·tanh(c)
Where it, ft, ot, and ct are values (i.e., input value, forgetting value, output value, cell state value) corresponding to the input gate, forgetting gate, output gate, and cell state at time step t of LSTM. As can be seen from the above formula, the hidden state ht is updated depending on the currently input word vector et, as well as the previous hidden state ht-1 and cell state ct-1. Wherein the cell state preserves a relatively long history of information. Therefore, with LSTM, the above (historical) information can be utilized into topic characterization. And BilSTM combines a forward LSTM and a backward LSTM, wherein the forward LSTM is iteratively processed from the first word of the topic to the current word, and the backward LSTM is iteratively processed from the last word to the current word (the forward LSTM and the backward LSTM both adopt the hts calculation formula). Thus, a contextual representation of the topic [ h1, h2, …, hn-1, hn ] can be obtained over the BilSTM network. I.e. concatenate the forward LSTM and backward LSTM results for each vocabulary:
Each hidden state can be viewed as a feature vector (i.e., a feature vector of a word) for the corresponding word xt.
(4) attention adjustment: the above-described process is the processing of the topic matrix [ x1, x2, …, xn-1, xn ], corresponding to the upper left-hand portion of FIG. 3. Step four would be to process the position information p1, p2, …, pn-1, pn. In the preprocessing, a vocabulary is given an initial attention weight PA or PN depending on whether it is from the answer. The initial attention weights (i.e. the attention weights mentioned above) are adjusted in two different ways to make it more reasonable. They are the following two implementation methods of the Attention attachment Layer.
attention zooming: the ratio between the answer vocabulary and the common vocabulary is dynamically adjusted according to the training process of the neural network so as to better represent the question and promote the labeling effect. The scaling between answer vocabulary and normal vocabulary is shown in fig. 4 a. For each vocabulary xi, their attention weights are adjusted by a scaling factor ws:
a=p×w
since Softmax normalization is performed after the Attention Adjustment Layer, the ratio between the answer vocabulary and the general vocabulary before and after performing Attention scaling may be changed to be remarkable because ws may be larger or smaller than 1 as the training result, so the ratio between them may be enlarged or reduced.
local connection adjustment: in addition, the vocabulary around the answer may also play an important role in reflecting the knowledge points of topic investigation. For example, the knowledge point label of the "emotional verb" in FIG. 2 is reflected by the word "can" preceding the answer "sting". Therefore, by partially connecting as shown in fig. 4b, the higher attention weight PA given to the answer during preprocessing is diffused into the words around it by the adjustment vector Wl, and the embodiment mainly considers the previous word and the next word. The adjustment process is as follows:
The local join adjustment process is just like the convolution operation, sliding the convolution kernel Wl over the problem in steps of 1. The adjusted attention weight ai for each vocabulary is the weighted sum of itself, the previous vocabulary, and the next vocabulary. The non-negative and three elements wl1, wl2, wl3 and a constraint of 1 are applied to the convolution kernel.
after the Attention Adjustment Layer is subjected to Attention scaling or local connection Adjustment, Softmax normalization operation is carried out to obtain the final Attention distribution [ s1, s2, …, sn-1, sn ].
(5) attention is paid to the application: the weighted Sum of topic context representations [ h1, h2, …, hn-1, hn ] is computed using the ultimate Attention distribution [ s1, s2, …, sn-1, sn ] as the feature vector of the topic, with the Attention Multiply Layer and Sum Layer in FIG. 3, i.e.:
r=s*h+s*h+…+s*h+s*h
where "+" is a scalar multiplication. As can be seen from the formula, the words given more attention contribute more information to the feature vector of the topic to more accurately characterize the topic.
(6) and (3) feature dimensionality reduction: in order to introduce more nonlinear information and generate better labeling effect, a full connection Layer (sense Layer) is used for feature dimension reduction.
q＝Relu(W·r+b)
where Relu is the activation function, holding a positive number value, and converting a non-positive number to 0. Thus, the final feature vector q of the title is obtained.
(7) Label prediction (knowledge point prediction or knowledge point label prediction): the number of the knowledge point labels is variable, so that the problem of multiple labels is solved. The multi-label problem is converted into a plurality of binary problems by adopting a strategy. And setting the number of neurons which is the same as that of the total labels on a Sigmoid Output Layer, fully connecting the final feature vector q of the question with the neurons, and activating by using a Sigmoid function to obtain the probability value of each label corresponding to the question. Meanwhile, in order to prevent overfitting, a dropout mechanism is arranged, and forward propagation of some nodes is blocked. The process is as follows:
Where is the corresponding element point-product computation, and u is the vector that determines which elements are to be blocked. The resulting vector z, whose element zi represents the probability that the problem examined the knowledge point yi. The predicted knowledge point label can then be obtained by methods such as setting the threshold Z ═ yi | zi ≧ α or sorting the probability values (see steps above for details).
(7) Loss function: in order to optimize parameters in the model during the training process, a labeled loss function is set for the whole model:
wherein, i (yi) indicates whether the question actually considers the knowledge point label yi, and the value is 0 or 1 (i.e. the above actual probability). The loss function is the cross entropy between the predicted probability vector [ z1, z2, …, zm-1, zm ] (i.e., the probability of investigation described above) and the actual probability vector [ I (y1), I (y2), …, I (ym-1), I (ym) ].
2. question bank labeling method based on keyword model
The above proposes a location-based attention model to capture more useful information to better characterize the topic. Another simpler model will now be proposed to capture the available information in another way, the structure of which is shown in fig. 5. Since the two models have many identical layers, the following will be described from their differences.
(1) pretreatment: the problem is preprocessed by integrating three parts of the single-choice problem. However, in the keyword-based model, the answer vocabulary is repeated several times and then filled in the question stem, as shown in the question input part of fig. 5. The title is then also denoted by one-hot, which is then converted into a word vector representation [ E1, E2, …, eu-1, eu ] by the word embedding matrix E.
(2) Topic characterization: to control variables, the context characterization of the topic is also obtained using BilSTM. But in contrast, this model no longer uses all time steps, i.e., all lexical outputs, of BilSTM. But instead the output hu of the last time step is used directly as the final characterization of the problem. Because the output of the BilSTM at each time step is based on the vocabulary of the time step and the current context information, the information of the vocabulary (keywords) of the answers can be extracted by the BilSTM for many times in a way of filling the vocabulary of the answers, thereby realizing more effective information capture. Meanwhile, the output of the last time step contains the information of the previous vocabulary, so the output hu of the last time step is taken as the final characteristic representation of the problem.
feature reduction after keyword-based modeling is similar to label prediction and location-based attention modeling, and is not described in detail.
3. Experiment of
for commercial reasons and for some other reasons, no well-constructed problem bank is ready for experimentation. Therefore, from the question bank website, English lists of primary schools and junior middle schools are crawled, and corresponding duplication removal and screening are carried out. Then, 5 students of the north teachers and big english professions mark knowledge points in the question bank, and 53 knowledge point marks are from national english course standards (experimental draft). The results of manual annotation are integrated by a 'rule of passing by half', and iterative modification is carried out on the controversial questions. And finally obtaining an English word choice question bank with good labels. Experiments were performed using 10-fold cross validation, with 9-fold for model training and 1-fold for testing.
hereinafter, the keyword-based model will be abbreviated as KBM. The attention model based on position has two different attention adjustment modes, namely attention zooming and local connection adjustment, which are abbreviated as PBAM (Scale) and PBAM (Localy) respectively. By way of example, their labeling for the title of FIG. 2 is shown in FIG. 6. It can be seen from the figure that the three models all produce correct labeling results though they can not completely predict the knowledge points of all the topics. Also due to the nature of the English question bank, PBAM (Scale) and PBAM (Locally) are almost focused on the answer vocabulary during training, but they still predict the "emotional verbs". This is because they can pass the "can" information to the "swing" hidden state during problem characterization, showing the advantage of using BilSTM for contextual characterization of topics.
The labeling performance of the two models was evaluated using the evaluation criteria Precision, Recall and F1-Score of the multi-label classification, and some conventional classification algorithms and topic models were used as comparison references. Wherein SVM (unigram) is a method of obtaining features using a bag-of-words model and obtaining labels using SVM. Both mlkn and KNN are k neighbors of a given topic, from which they determine the label for the given topic. Except that MLKNN determines the labels using maximum a posteriori probability, and KNN is determined by sorting the labels that appear in the neighbors by degree. The LDA-SVM utilizes the topic model LDA to obtain the topic characteristics of the topic, and then the SVM is trained to label. The results of their experiments on the English-language database are shown in Table 1. The experimental result shows that the knowledge point marking effect of the proposed model is obviously superior to that of some traditional machine learning methods. Compared with the optimal KNN in the traditional method, the method provided by the invention has the advantages that the index of F1-Score is respectively 6%, 5.4% and 9.4%. In addition, in some conventional methods, the difference between Precision and Recall values is large. This leads to a tendency that during the application process, the tags cannot be predicted completely or the predicted tags are not accurate enough. The method of the invention obtains balanced Precision and Recall values, which shows that the method of the invention is more effective and reliable.
of the three models pbam (scale), pbam (locally) and KBM, KBM achieves the best results, which may be due to two reasons. (1) In relation to the characteristics of the english question bank, the answers are particularly important to reflect the knowledge points of question investigation, while the word importance around the answers is not very large. And therefore less need for attention spreading. (2) Pbam (scale) and pbam (locally) always lose some information during attention-weighting of the hidden state of the vocabulary, making them behave slightly weaker than KBM.
The model of the present invention also showed superior levels in terms of the average performance of five annotators in performing annotation training.
TABLE 1 results of the experiment
| Method (model) | Precision | Recall | F1-Score |
| artificial annotation training | 0.8499 | 0.8486 | 0.8484 |
| SVM(unigram) | 0.4312 | 0.4639 | 0.4469 |
| MLKNN | 0.6531 | 0.5993 | 0.6251 |
| KNN | 0.6960 | 0.7609 | 0.7270 |
| LDA-SVM | 0.6412 | 0.6400 | 0.6409 |
| PBAM(Scale) | 0.7898 | 0.7849 | 0.7873 |
| PBAM(Locally) | 0.7816 | 0.7813 | 0.7811 |
| KBM | 0.8283 | 0.8145 | 0.8213 |
Fig. 7 is a schematic block diagram of an automatic question bank knowledge point labeling system according to an embodiment of the present invention.
fig. 7 shows an automatic question bank knowledge point labeling system, which includes:
A first obtaining unit 701, configured to pre-process each question in the question bank, and obtain a target question with an empty symbol;
a second obtaining unit 702, configured to obtain a word vector of each word in the target topic according to a position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
A calculating unit 703, configured to calculate, according to the word vector of each word and the word sequence information of the target title, the feature vector of the target title according to a preset policy;
And a determining unit 704, configured to determine a knowledge point for topic investigation according to the feature vector of the target topic and a preset knowledge point.
As a preferred embodiment, the first obtaining unit 701 is further configured to:
preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, the computing unit is further configured to:
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
And calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
As a preferred embodiment, the determining unit 704 is further configured to:
Calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
Confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
alternatively, the first and second electrodes may be,
sequencing the investigation probabilities from large to small;
And confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
the system and the method for automatically marking the question bank knowledge points are in one-to-one correspondence, so detailed description is omitted.
It should be noted that, in the respective components of the apparatus of the present invention, the components therein are logically divided according to the functions to be implemented, but the present invention is not limited thereto, and the respective components may be re-divided or combined as needed, for example, some components may be combined into a single component, or some components may be further decomposed into more sub-components.
it should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware.
the above embodiments are only suitable for illustrating the present invention and not limiting the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, so that all equivalent technical solutions also belong to the scope of the present invention, and the scope of the present invention should be defined by the claims.
Claims (8)
1. an automatic labeling method for question bank knowledge points is characterized by comprising the following steps:
Preprocessing each topic in a topic library, acquiring a target topic with an answer vocabulary filled in a null symbol, and acquiring a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title;
Determining knowledge points for topic investigation according to the feature vectors of the target topics and preset knowledge points;
the method comprises the following steps of preprocessing each question in a question bank, acquiring a target question with an empty symbol and filled with an answer vocabulary, and comprises the following steps:
Preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
Then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
and calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
2. The method of claim 1, wherein the adjustment vector is a scaling factor of an attention weight of each word;
Then, calculating the feature vector of the target topic according to the feature vector of each word, the attention weight of each word and a preset adjustment vector, including:
calculating the feature vector of the target object by formula (I) according to the feature vector of each word, the attention weight of each word and a preset adjustment vector
r ═ s1 ═ h1+ s2 ═ h2+ … + sn-1 × -hn-1 + sn × -hn formula (one)
wherein ai is pi × ws, r is a feature vector of the target topic, pi is an attention weight of the ith word, ws is a scaling factor of the attention weight of each word, h1., hn is a feature vector of the first word to the nth word respectively, and n is the total number of words of the target topic.
3. The method of claim 1, wherein the adjustment vector comprises three local adjustment factors, a first local adjustment factor, a second local adjustment factor, and a third local adjustment factor, wherein the three local adjustment factors are all non-negative numbers and have a sum of 1;
then, according to the feature vector of each word, the attention weight of each word and a preset adjustment vector, calculating the feature vector of the target topic by formula (I)
r ═ s1 ═ h1+ s2 ═ h2+ … + sn-1 × -hn-1 + sn × -hn formula (one)
wherein r is a feature vector of the target topic, p1 and p2 are attention weights of a first word and a second word respectively, pi-1, pi and pi +1 are attention weights of an i-1 th word, an i-th word and an i +1 th word respectively, pn-1 and pn are attention weights of an n-1 th word and an n-th word respectively, wl1, wl2 and wl3 are first local adjustment factors, second local adjustment factors and third local adjustment factors respectively, h1., hn is a feature vector of the first word to the n-th word respectively, and n is the total number of words of the target topic.
4. the method of claim 1, wherein preprocessing each topic in the topic library to obtain a target topic with an empty symbol for which an answer vocabulary is filled comprises:
preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
calculating a forward feature vector of each word and a backward feature vector of each word in the target topic according to the word vector of each word and the word sequence information of the target topic and a first preset strategy;
Calculating the feature vector of the target title according to the forward feature vector of each word and the backward feature vector of each word; the first preset strategy is as follows:
Calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of a first word according to the word vector, a preset forward initial characteristic vector and a preset forward initial cell state value of the first word in the target topic;
Calculating an input value, a forgetting value, an output value, a cell state value and a backward characteristic vector of the last word according to the word vector, a preset backward initial characteristic vector and a preset backward initial cell state value of the last word in the target topic;
sequentially and iteratively calculating the forward characteristic vector of each word from the first word to the current word according to the input value, the forgetting value, the output value, the cell state value, the forward characteristic vector of the first word and the word vector of each word between the first word and the current word, and obtaining the forward characteristic vector of each current word;
and sequentially carrying out iterative computation until the current word according to the sequence from the last word to the current word and obtaining the backward characteristic vector of each current word according to the input value, the forgetting value, the output value, the cell state value, the backward characteristic vector of the last word and the word vector of each word between the last word and the current word.
5. the method of claim 1, wherein preprocessing each topic in the topic library to obtain a target topic with an empty symbol for which an answer vocabulary is filled comprises:
preprocessing each question in the question bank, and acquiring a target question with repeatedly filled with multiple answer words at each preset null symbol;
then, calculating the feature vector of the target title according to a preset strategy and the word vector of each word and the target word sequence information of the target title, including:
Calculating an input value, a forgetting value, an output value, a cell state value and a forward characteristic vector of the first word according to the word vector of the first word, a preset forward initial characteristic vector and a preset forward initial cell state value;
Sequentially and iteratively calculating until the last word according to the input value, the forgetting value, the output value, the cell state value and the forward feature vector of the first word and the sequence from the first word to the last word to obtain the forward feature vector of the last word;
Calculating the backward characteristic vector of the last word according to the word vector of the last word in the target topic, a preset backward initial characteristic vector and a preset backward initial cell state value; and calculating the feature vector of the target topic according to the forward feature vector of the last word and the backward feature vector of the last word.
6. the method of claim 1, wherein determining knowledge points for topic exploration according to the feature vector and preset knowledge points of the target topic comprises:
Calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
Confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
alternatively, the first and second electrodes may be,
Sequencing the investigation probabilities from large to small;
And confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
7. An automatic marking system for question bank knowledge points is characterized by comprising:
The first acquisition unit is used for preprocessing each question in the question bank and acquiring a target question with an empty symbol filled with an answer vocabulary;
the second obtaining unit is used for obtaining a word vector of each word in the target topic according to the position of each word in the target topic and a preset word vector; the word vector is a vector containing semantic information;
the calculation unit is used for calculating the feature vector of the target title according to a preset strategy according to the word vector of each word and the target word sequence information of the target title;
the determining unit is used for determining knowledge points for topic investigation according to the feature vectors of the target topics and preset knowledge points;
wherein the first obtaining unit is further configured to:
Preprocessing each question in the question bank, and acquiring a target question with an answer vocabulary filled in each empty symbol position only once;
then, the computing unit is further configured to:
calculating a feature vector of each word according to the word vector of each word and the word sequence information of the target topic;
and calculating the feature vector of the target title according to the feature vector of each word, the attention weight of each word and a preset adjustment vector.
8. the system of claim 7, wherein the determination unit is further configured to:
calculating the investigation probability of the question to each preset knowledge point according to the feature vector of the target question and the preset knowledge points;
Confirming the knowledge points corresponding to the investigation probability which is greater than or equal to a preset threshold value in the investigation probability as the knowledge points of topic investigation;
alternatively, the first and second electrodes may be,
sequencing the investigation probabilities from large to small;
And confirming the knowledge points corresponding to the investigation probabilities of the first to the Nth bits in the sequencing result as the knowledge points for topic investigation, wherein N is a positive integer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710859784.7A CN107590127B (en) | 2017-09-21 | 2017-09-21 | Automatic marking method and system for question bank knowledge points |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710859784.7A CN107590127B (en) | 2017-09-21 | 2017-09-21 | Automatic marking method and system for question bank knowledge points |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107590127A CN107590127A (en) | 2018-01-16 |
| CN107590127B true CN107590127B (en) | 2019-12-06 |
Family
ID=61047032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710859784.7A Active CN107590127B (en) | 2017-09-21 | 2017-09-21 | Automatic marking method and system for question bank knowledge points |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107590127B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108182177A (en) * | 2018-01-24 | 2018-06-19 | 谢德刚 | A kind of mathematics knowledge-ID automation mask method and device |
| US20190236135A1 (en) * | 2018-01-30 | 2019-08-01 | Accenture Global Solutions Limited | Cross-lingual text classification |
| CN108536807B (en) * | 2018-04-04 | 2022-03-25 | 联想(北京)有限公司 | Information processing method and device |
| CN109766548B (en) * | 2018-12-29 | 2023-01-31 | 北京京师乐学教育科技有限公司 | Examination point identification method, examination point identification device, examination point identification equipment and computer storage medium |
| CN110263179A (en) * | 2019-06-12 | 2019-09-20 | 湖南酷得网络科技有限公司 | Learning path method for pushing, device, computer equipment and storage medium |
| CN110852071B (en) * | 2019-11-08 | 2023-10-24 | 科大讯飞股份有限公司 | Knowledge point detection method, device, equipment and readable storage medium |
| CN111475645B (en) * | 2020-03-17 | 2024-04-30 | 平安国际智慧城市科技股份有限公司 | Knowledge point labeling method, knowledge point labeling device and computer readable storage medium |
| CN111737402A (en) * | 2020-06-23 | 2020-10-02 | 上海松鼠课堂人工智能科技有限公司 | Question bank intelligent knowledge point marking system |
| CN112182237A (en) * | 2020-09-21 | 2021-01-05 | 深圳中兴网信科技有限公司 | Topic knowledge point association method, topic knowledge point association system and storage medium |
| CN113010687B (en) * | 2021-03-03 | 2023-02-03 | 广州视源电子科技股份有限公司 | Exercise label prediction method and device, storage medium and computer equipment |
| CN113420543B (en) * | 2021-05-11 | 2024-03-22 | 江苏大学 | Mathematical test question automatic labeling method based on improved Seq2Seq model |
| CN115186780B (en) * | 2022-09-14 | 2022-12-06 | 江西风向标智能科技有限公司 | Discipline knowledge point classification model training method, system, storage medium and equipment |
| CN117151083B (en) * | 2023-10-30 | 2024-04-19 | 中教畅享(北京)科技有限公司 | Calculation method for repetition rate in commodity title optimization |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183831A (en) * | 2015-08-31 | 2015-12-23 | 上海德唐数据科技有限公司 | Text classification method for different subject topics |
| CN105930319A (en) * | 2016-05-09 | 2016-09-07 | 北京新唐思创教育科技有限公司 | Methods and devices for establishing question knowledge point obtaining model and obtaining question knowledge point |
| CN106372058A (en) * | 2016-08-29 | 2017-02-01 | 中译语通科技（北京）有限公司 | Short text emotion factor extraction method and device based on deep learning |
| JP2017151838A (en) * | 2016-02-26 | 2017-08-31 | トヨタ自動車株式会社 | Topic estimation learning device and topic estimation learning method |
-
2017
- 2017-09-21 CN CN201710859784.7A patent/CN107590127B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105183831A (en) * | 2015-08-31 | 2015-12-23 | 上海德唐数据科技有限公司 | Text classification method for different subject topics |
| JP2017151838A (en) * | 2016-02-26 | 2017-08-31 | トヨタ自動車株式会社 | Topic estimation learning device and topic estimation learning method |
| CN105930319A (en) * | 2016-05-09 | 2016-09-07 | 北京新唐思创教育科技有限公司 | Methods and devices for establishing question knowledge point obtaining model and obtaining question knowledge point |
| CN106372058A (en) * | 2016-08-29 | 2017-02-01 | 中译语通科技（北京）有限公司 | Short text emotion factor extraction method and device based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| Rong Xiao et al.LSTM Based on the Classification of Emotion about User Evaluation on Shopping Site.《2016 International Conference on Identification，Information and Knowledge in the Internet of Things（IIKI）》.2016,第1-2页. * |
| 朱国进 等.基于深度学习的算法知识实体识别与发现.《智能计算机与应用》.2017,第7卷(第1期),第17-21页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107590127A (en) | 2018-01-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107590127B (en) | Automatic marking method and system for question bank knowledge points | |
| CN107562792B (en) | question-answer matching method based on deep learning | |
| CN110334354B (en) | Chinese relation extraction method | |
| CN112508334B (en) | Personalized paper grouping method and system integrating cognition characteristics and test question text information | |
| CN111881262B (en) | Text emotion analysis method based on multi-channel neural network | |
| US20210390873A1 (en) | Deep knowledge tracing with transformers | |
| US20220129621A1 (en) | Bert-based machine-learning tool for predicting emotional response to text | |
| CN110222178A (en) | Text sentiment classification method, device, electronic equipment and readable storage medium storing program for executing | |
| CN112905795A (en) | Text intention classification method, device and readable medium | |
| CN112257966B (en) | Model processing method and device, electronic equipment and storage medium | |
| CN112883714B (en) | ABSC task syntactic constraint method based on dependency graph convolution and transfer learning | |
| Alrajhi et al. | A multidimensional deep learner model of urgent instructor intervention need in MOOC forum posts | |
| CN113569001A (en) | Text processing method and device, computer equipment and computer readable storage medium | |
| CN112800190A (en) | Intent recognition and slot value filling joint prediction method based on Bert model | |
| CN114417851A (en) | Emotion analysis method based on keyword weighted information | |
| CN113486645A (en) | Text similarity detection method based on deep learning | |
| CN111145914B (en) | Method and device for determining text entity of lung cancer clinical disease seed bank | |
| CN114896386A (en) | Film comment semantic emotion analysis method and system based on BilSTM | |
| CN111914553A (en) | Financial information negative subject judgment method based on machine learning | |
| Yang | Application of English vocabulary presentation based on clustering in college English teaching | |
| Tashu et al. | Deep Learning Architecture for Automatic Essay Scoring | |
| Bai et al. | Gated character-aware convolutional neural network for effective automated essay scoring | |
| CN114169447B (en) | Event detection method based on self-attention convolution bidirectional gating cyclic unit network | |
| Zhao et al. | Determining the logical relation between two sentences by using the masked language model of BERT | |
| CN115129863A (en) | Intention recognition method, device, equipment, storage medium and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |