CN117852974A - Online evaluation score assessment method based on artificial intelligence - Google Patents
Online evaluation score assessment method based on artificial intelligence Download PDFInfo
- Publication number
- CN117852974A CN117852974A CN202410238866.XA CN202410238866A CN117852974A CN 117852974 A CN117852974 A CN 117852974A CN 202410238866 A CN202410238866 A CN 202410238866A CN 117852974 A CN117852974 A CN 117852974A
- Authority
- CN
- China
- Prior art keywords
- answer
- layer
- attention
- representing
- knowledge
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 22
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 15
- 230000008569 process Effects 0.000 claims abstract description 12
- 238000005259 measurement Methods 0.000 claims abstract description 4
- 239000013598 vector Substances 0.000 claims description 105
- 238000004364 calculation method Methods 0.000 claims description 41
- 239000011159 matrix material Substances 0.000 claims description 27
- 238000010606 normalization Methods 0.000 claims description 23
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 101100481876 Danio rerio pbk gene Proteins 0.000 claims description 3
- 101100481878 Mus musculus Pbk gene Proteins 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 230000002123 temporal effect Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000000691 measurement method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
- G06N5/025—Extracting rules from data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Strategic Management (AREA)
- Computational Linguistics (AREA)
- Educational Administration (AREA)
- Artificial Intelligence (AREA)
- Economics (AREA)
- Tourism & Hospitality (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Educational Technology (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Primary Health Care (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The invention relates to the technical field of education evaluation, and discloses an artificial intelligence-based online evaluation score evaluation method, which comprises the following steps: constructing a pre-training text understanding model to extract knowledge points of the standard answer data to obtain a standard answer knowledge point set; preprocessing the answer data of the students to obtain preprocessed answer data, preprocessing the preprocessed answer data into spoken language expression vocabulary text processing, and extracting answer points from the preprocessed answer data by using a pre-trained text understanding model to obtain a student answer point set; matching answer points in student answer data with standard answer knowledge points, and calculating coverage of the student answers to the standard answer knowledge points, wherein the implementation method of the knowledge point matching is text similarity measurement; and evaluating the answer data of the students according to the calculated knowledge point coverage. The invention can improve the evaluation efficiency, reduce the workload and time of manual scoring and reduce subjectivity and error possibly existing in the scoring process.
Description
Technical Field
The invention relates to the technical field of education evaluation, in particular to an artificial intelligence-based online evaluation score evaluation method.
Background
The online evaluation is a teaching and learning evaluation method through a network platform, and has the advantages of flexibility, high efficiency, expandability and the like. However, in online assessment, how to accurately evaluate the subjective answer score of students remains a challenge. Conventional assessment methods typically require manual intervention, are time and resource consuming, and are prone to subjectivity and inconsistency issues. Therefore, automated online assessment score assessment is a real problem that needs to be solved. Aiming at the problem, the method and the system for evaluating the online evaluation score based on the artificial intelligence automatically evaluate the subjective evaluation score of the students by utilizing the artificial intelligence technology, reduce the workload and time of the manual evaluation, and simultaneously reduce the subjectivity and error possibly existing in the scoring process.
Disclosure of Invention
In view of the above, the invention provides an artificial intelligence based on-line evaluation score evaluation method and system, which aims to: 1) According to the online evaluation score evaluation method based on artificial intelligence, standard answer knowledge points can be effectively extracted, student answer points can be extracted by combining a pre-training model, a text understanding technology and a text similarity measurement method, automatic evaluation is carried out on student answers through knowledge point coverage calculation, evaluation efficiency is improved, workload and time of manual evaluation are reduced, and subjectivity and error possibly existing in the scoring process are reduced; 2) By using a pre-training text understanding model, knowledge point extraction can be performed on standard answer data, answer points can be extracted from student answer data, key information in a text can be better understood and extracted, and a foundation is laid for automatic evaluation; 3) The sparse transducer encoder is used for extracting knowledge points and key points, so that the complexity of the model can be reduced while the global dependence is maintained, long-sequence answer data can be better obtained, the number of parameters required to be trained and stored is reduced, and the model is lighter and more efficient.
The invention provides an artificial intelligence-based online evaluation score evaluation method, which comprises the following steps:
s1: constructing a pre-training text understanding model to extract knowledge points of the standard answer data to obtain a standard answer knowledge point set;
s2: preprocessing the answer data of the students to obtain preprocessed answer data, wherein the preprocessing is text processing of spoken language expression vocabulary, and answer points of the preprocessed answer data are extracted by using a pre-trained text understanding model to obtain a student answer point set;
s3: matching answer points in student answer data with standard answer knowledge points, and calculating coverage of the student answers to the standard answer knowledge points, wherein the implementation method of the knowledge point matching is text similarity measurement;
s4: and evaluating the answer data of the students according to the calculated knowledge point coverage.
As a further improvement of the present invention:
optionally, the constructing a pre-training text understanding model in the step S1 includes:
the pre-training text understanding model comprises a transducer encoder layer and a knowledge point extraction layer, wherein the transducer encoder layer is composed of a plurality of improved transducer encoder blocks and is used for capturing context information in a text; the knowledge point extraction layer uses the fully connected layer to map the output of the transform encoder layer to a predefined knowledge point space.
Optionally, the improved transducer encoder is a sparse transducer encoder comprising an input embedding layer, a position encoding layer, an attention-based sparsification layer, a multi-headed self-attention layer, residual connection and layer normalization and feedforward neural network layer, wherein the input embedding layer converts each word or feature in the input sequence into an embedded vector representation; the position coding layer adds position codes for each position in the input sequence to reserve sequence information in the sequence; the attention-based sparsification layer reduces the temporal and spatial complexity of the attention computation by introducing sparsity; the multi-head self-attention layer performs multi-head attention calculation, and performs attention weighted summation by using the thinned attention weight; residual connection and layer normalization after each sub-layer, residual connection and layer normalization processing is applied; the feedforward neural network layer processes the attention mechanism output of each sequence position through two full-connection layers and an activation function;
the specific coding flow of the improved transducer coder is as follows:
s11: initializing an input embedding layer and a position coding layer to obtain an embedded representation of an input sequence;
s12: the embedded representation is input to a sparse layer based on attention for attention calculation, and sparse attention weight is obtained;
the attention score calculation formula is as follows:
；
the sparse attention weight calculation formula is:
；
where Q represents a query matrix, K represents a key matrix,representing the query matrix and key matrix dimensions, topk representing the k remaining attention weights with highest relevance to the query; normal represents a normalization operation;
s13: inputting the thinned attention weight into a multi-head self-attention layer, and carrying out weighted summation on the input by using the attention weight to obtain attention output;
the multi-head attention score calculation formula is as follows:
；
the multi-head attention weight calculation formula is as follows:
；
wherein,a query matrix representing the ith attention header, < ->A key matrix representing the ith attention header,matrix of values representing the ith attention header, < >>Attention weight calculation result representing the ith attention head, +.>Representing query matrix and key matrix dimensions;
s14: applying a residual connection and layer normalization on the attention output;
s15: inputting the output after residual connection and layer normalization to a feedforward neural network layer for nonlinear transformation;
s16: re-applying residual connection and layer normalization on the feedforward neural network layer;
s17: repeating steps S12-S16, stacking a plurality of encoder layers, and outputting a final encoded representation.
Optionally, preprocessing the answer data of the student in the step S2 to obtain preprocessed answer data, including:
the implementation method for preprocessing the student answer data to remove the spoken expression vocabulary in the answer data and replace the spoken expression vocabulary with the written expression vocabulary is an LSTM model, and the specific flow is as follows:
the encoder part in the LSTM model converts the spoken expression vocabulary into word vector representation through an embedding layer, then carries out sequence encoding through a plurality of LSTM layers, and inputs the word vector of the current time step and the hidden state of the previous time step to obtain the hidden state of the current time step; the hidden state of the last LSTM layer is the output of the encoder;
the decoder part in the LSTM model preprocesses the target written text and converts the target written text into word vector representation through an embedding layer; performing sequence decoding through a plurality of LSTM layers, and inputting a word vector of the current time step, a hidden state of the previous time step and the output of an encoder to obtain the hidden state of the current time step; transmitting the hidden state of the last LSTM layer to the full-connection layer, and generating the probability distribution of words of the next time step through a softmax function; stopping generating until the symbol is generated or the maximum sequence length is reached.
Optionally, in the step S3, matching the answer points in the student answer data with standard answer knowledge points includes:
carrying out vectorization processing on answer points and standard answer knowledge points in student answer data to respectively obtain answer point vectors and knowledge point vectors, matching according to the calculated vectors, and carrying out vectorization processing on the answer points and the standard answer knowledge points in the student answer data, wherein the vectorization processing specific flow is as follows:
s31: splitting each knowledge point word into characters, and mapping each character into word vectors at a character level through an embedding layer;
s32: the word vector sequence of the character level is input, the word vector representation of the forward context awareness is obtained by encoding the word vector sequence through a forward LSTM model, and a calculation formula is as follows:
；
wherein,hidden state vector representing forward LSTM at time t,/->Word vectors representing character levels input at time t;
s33: the word vector sequence of the character level is input, the word vector sequence is encoded through a reverse LSTM model, the word vector representation of the reverse context perception is obtained, and the calculation formula is as follows:
；
wherein,hidden state vector representing reverse LSTM at time t,/->Word vectors representing character levels input at time t;
s34: the generated word vectors of the forward LSTM model and the reverse LSTM model are fused to generate final context-related word vector representation, and the calculation formula is as follows:
；
wherein,fused word vector representation representing the mth word,/->And->K-th layer hidden state vector representing the mth knowledge point word forward and backward LSTM at time T, respectively,>normalization operation of the weights representing the k-th layer,/->Representing a scaling factor, L representing the number of layers;
s35: and carrying out vectorization processing on answer points and standard answer knowledge points in the student answer data according to the steps S31 to S34, calculating to obtain answer point vectors and knowledge point vectors, and carrying out similarity matching according to the calculated answer point vectors and the calculated knowledge point vectors.
Optionally, in the step S35, similarity matching is performed according to the answer gist vector and the knowledge point vector obtained by calculation, including:
and calculating the similarity of the answer point vectors and the knowledge point vectors by using the Euclidean distance, wherein the calculation formula is as follows:
；
wherein,and->The values of the knowledge point vector A and the answer gist vector B in the ith dimension are respectively represented.
Optionally, the step S3 calculates coverage of the standard answer knowledge points by the student answers, including:
if the similarity of the answer key point vector and the knowledge point vector exceeds the appointed preset, the fact that the student answer data contains preset knowledge points is explained, the number of hit knowledge points is increased by 1, the total number of hit knowledge points is counted, the coverage of the knowledge points is calculated according to the counted total number of hit knowledge points, and a calculation formula is as follows:
；
wherein,representing the number of hits, +.>Representing the total number of knowledge points in the standard answer.
Optionally, in the step S4, evaluating the answer data of the student according to the calculated coverage of the knowledge points includes:
；
wherein,representing knowledge point coverage, ++>Representing total score->Representing the score of the student's answer.
In order to solve the above-described problems, the present invention provides an electronic apparatus including:
a memory storing at least one instruction;
the communication interface is used for realizing the communication of the electronic equipment; and
And the processor executes the instructions stored in the memory to realize the artificial intelligence-based online evaluation score evaluation method.
In order to solve the above problems, the present invention further provides a computer readable storage medium having at least one instruction stored therein, the at least one instruction being executed by a processor in an electronic device to implement the above-described artificial intelligence-based online assessment score evaluation method.
Compared with the prior art, the invention provides an artificial intelligence-based online evaluation score evaluation method, which has the following advantages:
firstly, the online evaluation score evaluation method based on artificial intelligence is provided, standard answer knowledge points can be effectively extracted, student answer points can be extracted through combining a pre-training model, a text understanding technology and a text similarity measurement method, automatic evaluation is carried out on student answers through knowledge point coverage calculation, evaluation efficiency is improved, workload and time of manual evaluation are reduced, and subjectivity and error possibly existing in the scoring process are reduced.
Meanwhile, according to the scheme, knowledge point extraction can be carried out on standard answer data by using the pre-training text understanding model, answer points can be extracted from student answer data, key information in the text can be better understood and extracted, a foundation is laid for automatic evaluation, knowledge point and key point extraction is carried out by using a sparse transducer encoder, the complexity of the model can be reduced while global dependence is maintained, long-sequence answer data can be better processed, the number of parameters needing training and storage is reduced, and the model is lighter and efficient.
Drawings
FIG. 1 is a schematic flow chart of an artificial intelligence-based online evaluation score evaluation method according to an embodiment of the invention;
fig. 2 is a schematic structural diagram of an electronic device for implementing an artificial intelligence-based online evaluation score evaluation method according to an embodiment of the present invention.
In the figure: 1 an electronic device, 10 a processor, 11 a memory, 12 a program, 13 a communication interface.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The embodiment of the application provides an artificial intelligence-based online evaluation score evaluation method. The execution subject of the artificial intelligence-based online evaluation score evaluation method includes, but is not limited to, at least one of a server, a terminal and the like capable of being configured to execute the method provided by the embodiment of the application. In other words, the artificial intelligence based on-line evaluation score evaluation method may be performed by software or hardware installed in a terminal device or a server device, and the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Example 1:
s1: and constructing a pre-training text understanding model to extract knowledge points of the standard answer data to obtain a standard answer knowledge point set.
The step S1 is to construct a pre-training text understanding model, which comprises the following steps:
the pre-training text understanding model comprises a transducer encoder layer and a knowledge point extraction layer, wherein the transducer encoder layer is composed of a plurality of improved transducer encoder blocks and is used for capturing context information in a text; the knowledge point extraction layer uses the fully connected layer to map the output of the transform encoder layer to a predefined knowledge point space.
The improved transducer encoder is a sparse transducer encoder comprising an input embedding layer, a position encoding layer, an attention-based sparsification layer, a multi-headed self-attention layer, residual connection and layer normalization and feedforward neural network layer, wherein the input embedding layer converts each word or feature in an input sequence into an embedded vector representation; the position coding layer adds position codes for each position in the input sequence to reserve sequence information in the sequence; the attention-based sparsification layer reduces the temporal and spatial complexity of the attention computation by introducing sparsity; the multi-head self-attention layer performs multi-head attention calculation, and performs attention weighted summation by using the thinned attention weight; residual connection and layer normalization after each sub-layer, residual connection and layer normalization processing is applied; the feed forward neural network layer processes the attention mechanism output for each sequence position through two fully connected layers and an activation function.
The specific coding flow of the improved transducer coder is as follows:
s11: initializing an input embedding layer and a position coding layer to obtain an embedded representation of an input sequence;
s12: the embedded representation is input to a sparse layer based on attention for attention calculation, and sparse attention weight is obtained;
the attention score calculation formula is as follows:
；
the sparse attention weight calculation formula is:
；
where Q represents a query matrix, K represents a key matrix,representing the query matrix and key matrix dimensions, topk representing the k remaining attention weights with highest relevance to the query; normal represents a normalization operation;
s13: inputting the thinned attention weight into a multi-head self-attention layer, and carrying out weighted summation on the input by using the attention weight to obtain attention output;
the multi-head attention score calculation formula is as follows:
；
the multi-head attention weight calculation formula is as follows:
；
wherein,a query matrix representing the ith attention header, < ->A key matrix representing the ith attention header,matrix of values representing the ith attention header, < >>Attention weight calculation result representing the ith attention head, +.>Representing query matrix and key matrix dimensions;
s14: applying a residual connection and layer normalization on the attention output;
s15: inputting the output after residual connection and layer normalization to a feedforward neural network layer for nonlinear transformation;
s16: re-applying residual connection and layer normalization on the feedforward neural network layer;
s17: repeating steps S12-S16, stacking a plurality of encoder layers, and outputting a final encoded representation.
S2: preprocessing the answer data of the students to obtain preprocessed answer data, wherein the preprocessing is text processing of spoken language expression vocabulary, and answer points of the preprocessed answer data are extracted by using a pre-trained text understanding model to obtain a student answer point set.
And step S2, preprocessing the student answer data to obtain preprocessed answer data, wherein the step S comprises the following steps:
the implementation method for preprocessing the student answer data to remove the spoken expression vocabulary in the answer data and replace the spoken expression vocabulary with the written expression vocabulary is an LSTM model, and the specific flow is as follows:
the encoder part in the LSTM model converts the spoken expression vocabulary into word vector representation through an embedding layer, then carries out sequence encoding through a plurality of LSTM layers, and inputs the word vector of the current time step and the hidden state of the previous time step to obtain the hidden state of the current time step; the hidden state of the last LSTM layer is the output of the encoder;
the decoder part in the LSTM model preprocesses the target written text and converts the target written text into word vector representation through an embedding layer; performing sequence decoding through a plurality of LSTM layers, and inputting a word vector of the current time step, a hidden state of the previous time step and the output of an encoder to obtain the hidden state of the current time step; transmitting the hidden state of the last LSTM layer to the full-connection layer, and generating the probability distribution of words of the next time step through a softmax function; stopping generating until the symbol is generated or the maximum sequence length is reached.
S3: matching answer points in student answer data with standard answer knowledge points, and calculating coverage of the student answers to the standard answer knowledge points, wherein the implementation method of the knowledge point matching is text similarity measurement.
And in the step S3, matching the answer points in the student answer data with standard answer knowledge points, wherein the step comprises the following steps:
carrying out vectorization processing on answer points and standard answer knowledge points in student answer data to respectively obtain answer point vectors and knowledge point vectors, matching according to the calculated vectors, and carrying out vectorization processing on the answer points and the standard answer knowledge points in the student answer data, wherein the vectorization processing specific flow is as follows:
s31: splitting each knowledge point word into characters, and mapping each character into word vectors at a character level through an embedding layer;
s32: the word vector sequence of the character level is input, the word vector representation of the forward context awareness is obtained by encoding the word vector sequence through a forward LSTM model, and a calculation formula is as follows:
；
wherein,hidden state vector representing forward LSTM at time t,/->Word vectors representing character levels input at time t;
s33: the word vector sequence of the character level is input, the word vector sequence is encoded through a reverse LSTM model, the word vector representation of the reverse context perception is obtained, and the calculation formula is as follows:
；
wherein,hidden state vector representing reverse LSTM at time t,/->Word vectors representing character levels input at time t;
s34: the generated word vectors of the forward LSTM model and the reverse LSTM model are fused to generate final context-related word vector representation, and the calculation formula is as follows:
；
wherein,fused word vector representation representing the mth word,/->And->K-th layer hidden state vector representing the mth knowledge point word forward and backward LSTM at time T, respectively,>normalization operation of the weights representing the k-th layer,/->Representing a scaling factor, L representing the number of layers;
s35: and carrying out vectorization processing on answer points and standard answer knowledge points in the student answer data according to the steps S31 to S34, calculating to obtain answer point vectors and knowledge point vectors, and carrying out similarity matching according to the calculated answer point vectors and the calculated knowledge point vectors.
In the step S35, similarity matching is performed according to the answer key point vector and the knowledge point vector obtained by calculation, including:
and calculating the similarity of the answer point vectors and the knowledge point vectors by using the Euclidean distance, wherein the calculation formula is as follows:
；
wherein,and->The values of the knowledge point vector A and the answer gist vector B in the ith dimension are respectively represented.
And step S3, calculating the coverage of the student answers to the standard answer knowledge points, wherein the step S comprises the following steps:
if the similarity of the answer key point vector and the knowledge point vector exceeds the appointed preset, the fact that the student answer data contains preset knowledge points is explained, the number of hit knowledge points is increased by 1, the total number of hit knowledge points is counted, the knowledge point coverage of the question is calculated according to the counted total number of hit knowledge points, and a calculation formula is as follows:
；
wherein,representing the number of hits, +.>Representing the total number of knowledge points in the standard answer.
S4: and evaluating the answer data of the students according to the calculated knowledge point coverage.
And in the step S4, evaluating the answer data of the students according to the calculated knowledge point coverage, wherein the method comprises the following steps:
；
wherein,representing knowledge point coverage, ++>Representing total score->Representing the score of the student's answer.
Examples
Fig. 2 is a schematic structural diagram of an electronic device for implementing an artificial intelligence-based online evaluation score evaluation method according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication interface 13 and a bus, and may further comprise a computer program, such as program 12, stored in the memory 11 and executable on the processor 10.
The memory 11 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may in other embodiments also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only for storing application software installed in the electronic device 1 and various types of data, such as codes of the program 12, but also for temporarily storing data that has been output or is to be output.
The processor 10 may be comprised of integrated circuits in some embodiments, for example, a single packaged integrated circuit, or may be comprised of multiple integrated circuits packaged with the same or different functions, including one or more central processing units (Central Processing unit, CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects respective components of the entire electronic device using various interfaces and lines, executes or executes programs or modules (a program 12 for realizing an on-line evaluation score evaluation based on artificial intelligence, etc.) stored in the memory 11, and invokes data stored in the memory 11 to perform various functions of the electronic device 1 and process the data.
The communication interface 13 may comprise a wired interface and/or a wireless interface (e.g. WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the electronic device 1 and other electronic devices and to enable connection communication between internal components of the electronic device.
The bus may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.
Fig. 2 shows only an electronic device with components, it being understood by a person skilled in the art that the structure shown in fig. 2 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or may combine certain components, or may be arranged in different components.
For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for supplying power to each component, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, and the like are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The electronic device 1 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which will not be described herein.
The electronic device 1 may optionally further comprise a user interface, which may be a Display, an input unit, such as a Keyboard (Keyboard), or a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device 1 and for displaying a visual user interface.
It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.
The program 12 stored in the memory 11 of the electronic device 1 is a combination of instructions that, when executed in the processor 10, may implement:
constructing a pre-training text understanding model, and extracting knowledge points from the standard answer data to obtain a standard answer knowledge point set;
preprocessing the answer data of the students to obtain preprocessed answer data, wherein the preprocessing comprises word segmentation and stop word removal, and answering points of the preprocessed answer data are extracted by using a pre-trained text understanding model to obtain a student answering point set;
matching answer points in student answer data with standard answer knowledge points, and calculating coverage of the student answers to the standard answer knowledge points, wherein the text similarity measure is a main implementation method of knowledge point matching;
and evaluating the answer data of the students according to the calculated knowledge point coverage.
Specifically, the specific implementation method of the above instruction by the processor 10 may refer to descriptions of related steps in the corresponding embodiments of fig. 1 to 2, which are not repeated herein.
It should be noted that, the foregoing reference numerals of the embodiments of the present invention are merely for describing the embodiments, and do not represent the advantages and disadvantages of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. An artificial intelligence-based online evaluation score evaluation method is characterized by comprising the following steps:
s1: constructing a pre-training text understanding model to extract knowledge points of the standard answer data to obtain a standard answer knowledge point set;
s2: preprocessing the answer data of the students to obtain preprocessed answer data, wherein the preprocessing is text processing of spoken language expression vocabulary, and answer points of the preprocessed answer data are extracted by using a pre-trained text understanding model to obtain a student answer point set;
s3: matching answer points in student answer data with standard answer knowledge points, and calculating coverage of the student answers to the standard answer knowledge points, wherein the implementation method of the knowledge point matching is text similarity measurement;
s4: and evaluating the answer data of the students according to the calculated knowledge point coverage.
2. The method for evaluating an online evaluation score based on artificial intelligence according to claim 1, wherein the constructing a pre-trained text understanding model in step S1 comprises:
the pre-training text understanding model comprises a transducer encoder layer and a knowledge point extraction layer, wherein the transducer encoder layer is composed of a plurality of improved transducer encoder blocks and is used for capturing context information in a text; the knowledge point extraction layer uses the fully connected layer to map the output of the transform encoder layer to a predefined knowledge point space.
3. The method for evaluating an online evaluation score based on artificial intelligence according to claim 2,
the improved transducer encoder is a sparse transducer encoder comprising an input embedding layer, a position encoding layer, an attention-based sparsification layer, a multi-headed self-attention layer, residual connection and layer normalization and feedforward neural network layer, wherein the input embedding layer converts each word or feature in an input sequence into an embedded vector representation; the position coding layer adds position codes for each position in the input sequence to reserve sequence information in the sequence; the attention-based sparsification layer reduces the temporal and spatial complexity of the attention computation by introducing sparsity; the multi-head self-attention layer performs multi-head attention calculation, and performs attention weighted summation by using the thinned attention weight; residual connection and layer normalization after each sub-layer, residual connection and layer normalization processing is applied; the feedforward neural network layer processes the attention mechanism output of each sequence position through two full-connection layers and an activation function;
the specific coding flow of the improved transducer coder is as follows:
s11: initializing an input embedding layer and a position coding layer to obtain an embedded representation of an input sequence;
s12: the embedded representation is input to a sparse layer based on attention for attention calculation, and sparse attention weight is obtained;
the attention score calculation formula is as follows:
；
the sparse attention weight calculation formula is:
；
where Q represents a query matrix, K represents a key matrix,representing the query matrix and key matrix dimensions, topk representing the k remaining attention weights with highest relevance to the query; normal represents a normalization operation;
s13: inputting the thinned attention weight into a multi-head self-attention layer, and carrying out weighted summation on the input by using the attention weight to obtain attention output;
the multi-head attention score calculation formula is as follows:
；
the multi-head attention weight calculation formula is as follows:
；
wherein,a query matrix representing the ith attention header, < ->Key matrix representing the ith attention head,/-j>Matrix of values representing the ith attention header, < >>Attention weight calculation result representing the ith attention head, +.>Representing query matrix and key matrix dimensions;
s14: applying a residual connection and layer normalization on the attention output;
s15: inputting the output after residual connection and layer normalization to a feedforward neural network layer for nonlinear transformation;
s16: re-applying residual connection and layer normalization on the feedforward neural network layer;
s17: repeating steps S12-S16, stacking a plurality of encoder layers, and outputting a final encoded representation.
4. The method for evaluating an online evaluation score based on artificial intelligence according to claim 1, wherein the step S2 of preprocessing the answer data of the student to obtain preprocessed answer data comprises:
the implementation method for preprocessing the student answer data to remove the spoken expression vocabulary in the answer data and replace the spoken expression vocabulary with the written expression vocabulary is an LSTM model, and the specific flow is as follows:
the encoder part in the LSTM model converts the spoken expression vocabulary into word vector representation through an embedding layer, then carries out sequence encoding through a plurality of LSTM layers, and inputs the word vector of the current time step and the hidden state of the previous time step to obtain the hidden state of the current time step; the hidden state of the last LSTM layer is the output of the encoder;
the decoder part in the LSTM model preprocesses the target written text and converts the target written text into word vector representation through an embedding layer; performing sequence decoding through a plurality of LSTM layers, and inputting a word vector of the current time step, a hidden state of the previous time step and the output of an encoder to obtain the hidden state of the current time step; transmitting the hidden state of the last LSTM layer to the full-connection layer, and generating the probability distribution of words of the next time step through a softmax function; stopping generating until the symbol is generated or the maximum sequence length is reached.
5. The method for evaluating an online evaluation score based on artificial intelligence according to claim 1, wherein the step S3 of matching answer points in student answer data with standard answer knowledge points comprises:
carrying out vectorization processing on answer points and standard answer knowledge points in student answer data to respectively obtain answer point vectors and knowledge point vectors, matching according to the calculated vectors, and carrying out vectorization processing on the answer points and the standard answer knowledge points in the student answer data, wherein the vectorization processing specific flow is as follows:
s31: splitting each knowledge point word into characters, and mapping each character into word vectors at a character level through an embedding layer;
s32: the word vector sequence of the character level is input, the word vector representation of the forward context awareness is obtained by encoding the word vector sequence through a forward LSTM model, and a calculation formula is as follows:
；
wherein,hidden state vector representing forward LSTM at time t,/->Word vectors representing character levels input at time t;
s33: the word vector sequence of the character level is input, the word vector sequence is encoded through a reverse LSTM model, the word vector representation of the reverse context perception is obtained, and the calculation formula is as follows:
；
wherein,hidden state vector representing reverse LSTM at time t,/->Word vectors representing character levels input at time t;
s34: the generated word vectors of the forward LSTM model and the reverse LSTM model are fused to generate final context-related word vector representation, and the calculation formula is as follows:
；
wherein,fused word vector representation representing the mth word,/->And->K-th layer hidden state vector representing the mth knowledge point word forward and backward LSTM at time T, respectively,>representing the weight of the kth layerNormalization operation performed,/->Representing a scaling factor, L representing the number of layers;
s35: and carrying out vectorization processing on answer points and standard answer knowledge points in the student answer data according to the steps S31 to S34, calculating to obtain answer point vectors and knowledge point vectors, and carrying out similarity matching according to the calculated answer point vectors and the calculated knowledge point vectors.
6. The method for evaluating an online evaluation score based on artificial intelligence according to claim 5, wherein the step S35 of similarity matching is performed according to the answer point vector and the knowledge point vector obtained by calculation, and the method comprises the steps of:
and calculating the similarity of the answer point vectors and the knowledge point vectors by using the Euclidean distance, wherein the calculation formula is as follows:
；
wherein,and->The values of the knowledge point vector A and the answer gist vector B in the ith dimension are respectively represented.
7. The artificial intelligence based on-line evaluation score evaluation method of claim 1, wherein the step S3 of calculating coverage of the standard answer knowledge points by the student answers comprises:
if the similarity of the answer key point vector and the knowledge point vector exceeds the appointed preset, the fact that the student answer data contains preset knowledge points is explained, the number of hit knowledge points is increased by 1, the total number of hit knowledge points is counted, the coverage of the knowledge points is calculated according to the counted total number of hit knowledge points, and a calculation formula is as follows:
；
wherein,representing the number of hits, +.>Representing the total number of knowledge points in the standard answer.
8. The method for evaluating an online evaluation score based on artificial intelligence according to claim 1, wherein the step S4 of evaluating student answer data according to the calculated knowledge point coverage comprises:
；
wherein,representing knowledge point coverage, ++>Representing total score->Representing the score of the student's answer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410238866.XA CN117852974B (en) | 2024-03-04 | 2024-03-04 | Online evaluation score assessment method based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410238866.XA CN117852974B (en) | 2024-03-04 | 2024-03-04 | Online evaluation score assessment method based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117852974A true CN117852974A (en) | 2024-04-09 |
| CN117852974B CN117852974B (en) | 2024-06-21 |
Family
ID=90529377
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410238866.XA Active CN117852974B (en) | 2024-03-04 | 2024-03-04 | Online evaluation score assessment method based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117852974B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118261163A (en) * | 2024-05-30 | 2024-06-28 | 杭州容博教育科技有限公司 | Intelligent evaluation report generation method and system based on transformer structure |
| CN118469408A (en) * | 2024-07-11 | 2024-08-09 | 杭州和利时自动化有限公司 | Thermal power plant operator performance data processing method and system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727778A (en) * | 2019-10-15 | 2020-01-24 | 大连中河科技有限公司 | Intelligent question-answering system for tax affairs |
| CN112100356A (en) * | 2020-09-17 | 2020-12-18 | 武汉纺织大学 | Knowledge base question-answer entity linking method and system based on similarity |
| CN113704437A (en) * | 2021-09-03 | 2021-11-26 | 重庆邮电大学 | Knowledge base question-answering method integrating multi-head attention mechanism and relative position coding |
| CN114492759A (en) * | 2021-02-05 | 2022-05-13 | 谷歌有限责任公司 | Sparse attention neural network |
| CN115906855A (en) * | 2022-12-27 | 2023-04-04 | 浙江邦盛科技股份有限公司 | Word information fused Chinese address named entity recognition method and device |
| CN116432019A (en) * | 2022-12-09 | 2023-07-14 | 华为技术有限公司 | Data processing method and related equipment |
| CN116860947A (en) * | 2023-07-31 | 2023-10-10 | 西安交通大学 | Text reading and understanding oriented selection question generation method, system and storage medium |
| CN117152416A (en) * | 2023-09-01 | 2023-12-01 | 淮阴工学院 | Sparse attention target detection method based on DETR improved model |
| CN117474094A (en) * | 2023-12-22 | 2024-01-30 | 云南师范大学 | Knowledge tracking method based on fusion domain features of Transformer |
-
2024
- 2024-03-04 CN CN202410238866.XA patent/CN117852974B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110727778A (en) * | 2019-10-15 | 2020-01-24 | 大连中河科技有限公司 | Intelligent question-answering system for tax affairs |
| CN112100356A (en) * | 2020-09-17 | 2020-12-18 | 武汉纺织大学 | Knowledge base question-answer entity linking method and system based on similarity |
| CN114492759A (en) * | 2021-02-05 | 2022-05-13 | 谷歌有限责任公司 | Sparse attention neural network |
| CN113704437A (en) * | 2021-09-03 | 2021-11-26 | 重庆邮电大学 | Knowledge base question-answering method integrating multi-head attention mechanism and relative position coding |
| CN116432019A (en) * | 2022-12-09 | 2023-07-14 | 华为技术有限公司 | Data processing method and related equipment |
| CN115906855A (en) * | 2022-12-27 | 2023-04-04 | 浙江邦盛科技股份有限公司 | Word information fused Chinese address named entity recognition method and device |
| CN116860947A (en) * | 2023-07-31 | 2023-10-10 | 西安交通大学 | Text reading and understanding oriented selection question generation method, system and storage medium |
| CN117152416A (en) * | 2023-09-01 | 2023-12-01 | 淮阴工学院 | Sparse attention target detection method based on DETR improved model |
| CN117474094A (en) * | 2023-12-22 | 2024-01-30 | 云南师范大学 | Knowledge tracking method based on fusion domain features of Transformer |
Non-Patent Citations (2)
| Title |
|---|
| 余传明;王曼怡;林虹君;朱星宇;黄婷婷;安璐;: "基于深度学习的词汇表示模型对比研究", 数据分析与知识发现, no. 08, 31 December 2020 (2020-12-31) * |
| 侯丽仙;李艳玲;林民;李成城;: "融合多约束条件的意图和语义槽填充联合识别", 计算机科学与探索, no. 09, 31 December 2020 (2020-12-31) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118261163A (en) * | 2024-05-30 | 2024-06-28 | 杭州容博教育科技有限公司 | Intelligent evaluation report generation method and system based on transformer structure |
| CN118469408A (en) * | 2024-07-11 | 2024-08-09 | 杭州和利时自动化有限公司 | Thermal power plant operator performance data processing method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117852974B (en) | 2024-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117852974B (en) | Online evaluation score assessment method based on artificial intelligence | |
| US11620449B2 (en) | Method for machine reading comprehension | |
| CN111554268A (en) | Language identification method based on language model, text classification method and device | |
| CN114822812B (en) | Role dialogue simulation method, device, equipment and storage medium | |
| CN114461777B (en) | Intelligent question-answering method, device, equipment and storage medium | |
| CN113821622B (en) | Answer retrieval method and device based on artificial intelligence, electronic equipment and medium | |
| CN107832300A (en) | Towards minimally invasive medical field text snippet generation method and device | |
| CN117121015A (en) | Multimodal, less-hair learning using frozen language models | |
| CN113327279A (en) | Point cloud data processing method and device, computer equipment and storage medium | |
| CN113807973B (en) | Text error correction method, apparatus, electronic device and computer readable storage medium | |
| CN113705191A (en) | Method, device and equipment for generating sample statement and storage medium | |
| CN116403161A (en) | Intelligent fire monitoring system based on image recognition | |
| CN116450829A (en) | Medical text classification method, device, equipment and medium | |
| CN116827685B (en) | Dynamic defense strategy method of micro-service system based on deep reinforcement learning | |
| CN116341646A (en) | Pretraining method and device of Bert model, electronic equipment and storage medium | |
| CN108596687B (en) | Advertisement targeted delivery method based on Hash learning | |
| CN116665878A (en) | Intelligent inquiry method, device, equipment and storage medium for improving accumulated errors | |
| CN113469237B (en) | User intention recognition method, device, electronic equipment and storage medium | |
| CN115617975A (en) | Intention identification method and device for few-sample and multi-turn conversations | |
| CN115759052A (en) | Text error correction method and device, electronic equipment and storage medium | |
| CN114580738A (en) | Social media crisis event prediction method and system | |
| CN114006986A (en) | Outbound call compliance early warning method, device, equipment and storage medium | |
| CN116578690B (en) | Insurance customer service method and system based on artificial intelligence | |
| CN114546882B (en) | Intelligent question-answering system testing method and device, electronic equipment and storage medium | |
| CN113988081B (en) | Similarity problem matching method based on Coarse2Fine network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |