Disclosure of Invention
Therefore, the embodiment of the invention provides an evaluation method based on a CO-MIRT algorithm model, so as to solve the problems of complicated evaluation content, poor evaluation efficiency and poor accuracy caused by incapability of balancing measurement errors, measurement quantity and the number of knowledge points to be measured in the prior art.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
in a first aspect, an embodiment of the present invention provides an evaluation method based on a CO-MIRT algorithm model, including: obtaining basic parameter information of an evaluation event; compiling the basic parameter information into a knowledge point hierarchy set and a test question information set; setting a corresponding test question ID list and a corresponding student answer list according to the knowledge point hierarchy set and the test question information set; and after the target object finishes answering, performing evaluation analysis by adopting a preset CO-MIRT algorithm model based on the input test question ID list and the student answer list, and outputting a test grade analysis result.
Further, the basic parameter information includes: the method comprises the steps of acquiring a knowledge map containing multi-level knowledge points, a to-be-detected knowledge point set, a test question pool and test question parameters; at least one knowledge point to be measured is measured for each question in the test question pool; the test question parameters comprise at least one of test question Q matrix parameters, discrimination parameters and difficulty parameters.
Furthermore, the contents of the knowledge points contained in the bottom knowledge points of the knowledge map, the to-be-detected knowledge point set and the test question matrix parameters are consistent.
Furthermore, the knowledge point hierarchy set is used for drawing each layer of knowledge points in the three-dimensional knowledge graph and subordinate knowledge points of each layer of knowledge points into an adjacent matrix, and arranging the adjacent matrices according to a hierarchy sequence; and the test question Q matrix, the discrimination and the difficulty in the test question information set are sorted according to the test question ID.
Further, the test question ID list records the IDs of all test questions answered by the current target object; and the answer list records the result of right or wrong answer of the current target object to each question.
Further, the CO-MIRT algorithm model comprises: the method comprises a feedforward layer constructed based on an adjacency matrix, a full connection layer constructed aiming at a single-layer knowledge point, an output layer accessed to a preset MIRT model and a control layer designed based on a preset algorithm.
Further, the CO-MIRT algorithm model further comprises a telescopic layer designed based on a preset algorithm.
In a second aspect, an embodiment of the present invention further provides an evaluation device based on a CO-MIRT algorithm model, including: the parameter information acquisition unit is used for acquiring basic parameter information of the evaluation event; the parameter information compiling unit is used for compiling the basic parameter information into a knowledge point hierarchy set and a test question information set; the setting list unit is used for setting a test question ID list and a student answer list according to the knowledge point hierarchy set and the test question information set; and the evaluation analysis unit is used for carrying out evaluation analysis on the basis of the input test question ID list and the student answer list by adopting a preset CO-MIRT algorithm model after the target object finishes answering, and outputting an evaluation analysis result.
Further, the basic parameter information includes: the method comprises the steps of acquiring a knowledge map containing multi-level knowledge points, a to-be-detected knowledge point set, a test question pool and test question parameters; at least one knowledge point to be measured is measured for each question in the test question pool; the test question parameters comprise at least one of test question Q matrix parameters, discrimination parameters and difficulty parameters.
Furthermore, the contents of the knowledge points contained in the bottom knowledge points of the knowledge map, the set of knowledge points to be tested and the parameters of the test question Q matrix are consistent.
Furthermore, the knowledge point hierarchy set is used for drawing each layer of knowledge points in the knowledge graph and subordinate knowledge points of each layer of knowledge points into an adjacent matrix and discharging the adjacent matrixes according to a hierarchy sequence; and the test question Q matrix, the discrimination and the difficulty in the test question information set are sorted according to the test question ID.
Further, the test question ID list records the IDs of all test questions answered by the current target object; and the answer list records the result of right or wrong answer of the current target object to each question.
Further, the CO-MIRT algorithm model comprises: the method comprises a feedforward layer constructed based on an adjacency matrix, a full connection layer constructed aiming at a single-layer knowledge point, an output layer accessed to a preset MIRT model and a control layer designed based on a preset algorithm.
Further, the CO-MIRT algorithm model further comprises a telescopic layer designed based on a preset algorithm.
In a third aspect, an embodiment of the present invention further provides an electronic device, including: a processor and a memory; the memory is used for storing a program of the evaluation method based on the CO-MIRT algorithm model, and the electronic equipment is powered on and executes the program of the evaluation method based on the CO-MIRT algorithm model through the processor, so that any one of the evaluation methods based on the CO-MIRT algorithm model is executed.
In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, where the computer-readable storage medium contains one or more program instructions for executing any one of the above evaluation methods based on the CO-MIRT algorithm model.
By adopting the evaluation method based on the CO-MIRT algorithm model, balance among the measurement error, the measurement quantity and the number of the knowledge points to be measured can be considered, evaluation content is simplified, event evaluation efficiency is improved, high precision of an evaluation effect can be realized, and use experience of a user is effectively improved.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes an embodiment of the evaluation method based on the CO-MIRT algorithm model in detail. As shown in fig. 1, which is a flowchart of an evaluation method based on a CO-MIRT algorithm model according to an embodiment of the present invention, a specific implementation process includes the following steps:
step S101: and obtaining basic parameter information of the evaluation event.
Specifically, the basic parameter information includes: the method comprises a knowledge map of multi-level knowledge points, a set of knowledge points to be detected, a test question pool, test question parameters and the like. At least one knowledge point to be measured is measured for each question in the test question pool; the test question parameters comprise at least one of test question Q matrix parameters, discrimination parameters, difficulty parameters and the like. It should be noted that, in the specific implementation process, the contents of the knowledge points included in the bottom knowledge points of the knowledge map, the set of knowledge points to be tested, and the parameters of the test question Q matrix are consistent.
Step S102: and compiling the basic parameter information into a knowledge point hierarchy set and a test question information set.
After obtaining the basic parameter information of the evaluation event in step S101, the basic parameter information may be further compiled in this step to obtain a corresponding knowledge point hierarchy set and test question information set.
Specifically, the knowledge point hierarchy set is obtained by drawing each layer of knowledge points in the three-dimensional knowledge graph and subordinate knowledge points of each layer of knowledge points into an adjacent matrix, and arranging the adjacent matrices according to a hierarchy sequence; and the test question Q matrix, the discrimination and the difficulty in the test question information set are sorted according to the test question ID. Wherein the size of the adjacency matrix is as follows: [ num _ parts, num _ child ]; num _ entries represents the number of parent knowledge points, num _ child represents the number of child knowledge points, if a relationship exists between the parent knowledge points and the child knowledge points, the number of the parent knowledge points is marked as 1, otherwise, the number of the child knowledge points is marked as 0; the initial adjacency matrix is given as [1, num _ child ], where num _ child is the number of first-layer knowledge points.
Step S103: and setting a corresponding test question ID list and a corresponding student answer list according to the knowledge point hierarchy set and the test question information set.
After the knowledge point hierarchy set and the test question information set are compiled in step S102, a corresponding test question ID list and student answer list may be set in this step.
In the embodiment of the invention, because the evaluation event adopts a self-adaptive test strategy, the initial question is set as a question for investigating a certain knowledge point to be tested and the difficulty is close to 0. Furthermore, a corresponding test question ID list and a corresponding student answer list are set according to the knowledge point hierarchy set and the test question information set. Wherein, the test question ID list records the IDs of all test questions answered by the current target object; the answer list records the answer or wrong answer result of the current target object for each question, and the answer list records the answer or wrong answer result as 1 or 0.
Step S104: and after the target object finishes answering, performing evaluation analysis by adopting a preset CO-MIRT algorithm model based on the input test question ID list and the student answer list, and outputting an evaluation analysis result.
After the test question ID list and the student answer list are obtained in step S103, the test question ID list and the student answer list may be input to a preset CO-MIRT algorithm model in this step, and evaluation analysis may be performed based on the CO-MIRT algorithm model.
In the embodiment of the invention, a CO-MIRT (cooperative multi-dimensional project response theory) algorithm model is a technical innovation of an MIRT (multi-dimensional project response theory) model by utilizing a deep learning technology, and aims to solve the balance problem among the number of answers, measurement errors and user experience in industrial application. Wherein, the CO-MIRT algorithm model comprises: the method comprises a feedforward layer constructed based on an adjacency matrix, a full connection layer constructed aiming at a single-layer knowledge point, and an output layer accessed to a preset MIRT model. It should be noted that, considering that under the condition of a small number of subjects, two contradictory problems occur, namely: the operation speed can be improved by using a gradient descent method to complete parameter estimation, but the output result tends to move to an extreme value, so that a control layer is designed in a CO-MIRT algorithm model based on a preset algorithm to prevent the situation from occurring; however, in the implementation process, the control of the extreme value may cause the values at both sides of the data distribution (usually representing the target objects with better or worse performance) to be severely compressed, so in order to compensate the consequences caused by the control layer, the telescopic layer is designed by using the value of the historical data and based on the preset algorithm, so as to achieve high precision of the evaluation effect. The target object refers to a student aimed at the evaluation event.
Regarding the feedforward layer, the performance of the target object (such as a student) under each subdivision knowledge point is not completely independent, but has a certain mathematical relationship, and the relationship can be divided into the relationship between a child knowledge point and a parent knowledge point in the longitudinal direction and the relationship between a child knowledge point and a brother knowledge point in the transverse direction. The mathematical relationship is expressed as follows, wherein chilren represents the level of the child knowledge point, parentes represents the level of the parent knowledge point, w represents the weight, and b represents the bias term:
children=f(parents,w,b);children=g(children,w,b)。
based on the theoretical assumption, a preset algorithm can be used for drawing an adjacent matrix (also called an A matrix) diagram according to the parent-child relationship in the knowledge graph, and the CO-MIRT algorithm model construction is completed on the basis. Suppose that the feedforward layer coexists in the L layer, L ═ L1,l2,...,ls}lsRepresents the s-th layer of the L layers. Wherein, the firsts-1Layer is the firstsParent node of the layer, < th > lsLayer is the firsts-1A child node of a layer. FirstsThe matrix size of the layer is m x n, wherein m is the number of knowledge points of the parent node, and n is the number of knowledge points of the child node. If the relation exists between the mth father node and the nth child node, the relation is marked as 1, otherwise, the relation is marked as 0. theta denotes the capability value of the target object, thetasThe s-th level capability vector representing the current target object (note: s here is the same meaning as s in the knowledge-graph). The specific mathematical relationships are not described in detail herein.
With respect to the fully-connected layer, the fully-connected layer is the process of performing a fully-connected computation for each layer output in the feedforward layer. Wherein w represents a weight, b represents a bias term, thetasL < th > indicating the current studentsThe energy vector of the layer. Initialization of w and b is given randomly and calculated via gradient descent, thetasIs formed byThe feedforward layer is calculated. It should be noted that the fully-connected layer output for the underlying knowledge points (i.e., the lowest knowledge points of the knowledge graph) is the student's ability estimate that we expect to obtain.
The specific mathematical relationship is as follows:
the weight calculation algorithm is as follows: weights=softmax(weights);
The capacity calculation algorithm is as follows: theta (theta)s=thetas-1οweights+betas;
The capability activation algorithm is: theta (theta)s=tanh(thetas);thetac=thetac。
Wherein c represents the underlying knowledge point and does not participate in activation.
Regarding the output layer, the output layer is to combine the output (estimation value of target object capability) of the last layer of the fully-connected layer with the test question parameters (Q matrix, discrimination a and difficulty b) to form a logic, so as to complete the calculation of the loss function. Wherein c represents the bottom knowledge point in the knowledge graph, j represents the jth topic, qjQ matrix, a, representing the jth topicjShows the degree of discrimination of the jth topic, bjIndicating the difficulty of the jth topic. The specific mathematical relationships are not detailed herein.
Regarding the control layer, the purpose of the control layer is to prevent extreme values from occurring in the output results. In the case of a small number of subjects, the gradient descent method causes thetac(c represents the bottom layer of the knowledge graph) is towards the extreme end, and experiments show that the use of tanh can effectively inhibit the phenomenon, but the value range of tanh is (-1,1), and the target objects with better or poorer performances can be actually discarded. Therefore, a control layer is designed based on a preset algorithm, and the results of tanh and non-tanh are comprehensively considered to control the output result.
The specific mathematical relationship is as follows:
a control vector, where k represents the number of underlying knowledge points:
control=[w1,w2,...,wk](ii) a Wherein k represents the number of underlying knowledge points;
control sigmoid (control); the control calculation is:
thetac=tanh(thetac)·control+thetac·(1-control)。
with respect to the telescopic layer, the purpose of the telescopic layer is to compensate for the problem of the control layer causing the target object capability vector to be squeezed. The main logic of the telescopic layer is to multiply theta by matrixc(matrix size 1 x k, k denotes the number of underlying knowledge points) compression to thetatotal(thetatotalIs a numerical value representing the overall ability of the target object in the discipline). Then, the history data such as the joint survey data is revised in advance about thetatotalAnd the manner in which the product operates directs the target object to fill in its level in the grade, thereby obtaining thetapre(thetapreIndicating a priori capability). Finally, we calculate thetatotalAnd thetapreThe mean square error of the method achieves the purpose of scaling.
The specific mathematical relationship is as follows:
and (3) error calculation:
punish_total=(thetatotal-thetapre)2
in addition, it should be noted that the loss function is an cross-entropy loss function commonly used in the industry, and the penalty term is for thetacL2 regularization, and the output of the above error calculation equation.
In a specific implementation process, the specific implementation process of performing evaluation analysis based on the input test question ID list and student answer list by using a preset CO-MIRT algorithm model may include the following steps: step one, if the initial target object capability is 0, completing a calculation task of a feedforward layer (namely calculating with an adjacent matrix in a knowledge point hierarchy set), wherein a calculation of a full connection layer is performed in each feedforward layer; secondly, after the calculation of the feedforward layer is completed, the calculation of two full-connected layers is completed, wherein the calculation of the control layer is carried out in the last full-connected layer and is marked as theta, namely the estimated value of the target object capacity; thirdly, calculating the output result of the full connection layer (the question information comes from the test question information set); fourthly, constructing a cross entropy loss function for the result of the output layer, and adding a penalty term (including calculation of the telescopic layer and regularization of theta); fifthly, iterative computation is carried out by using a gradient descent method; and sixthly, performing accumulation calculation on the test question matrix (namely the Q matrix) according to the test question ID list, determining a knowledge point corresponding to the minimum accumulation value, selecting a question which is used for examining the knowledge point and has the maximum fisher information amount (using theta) as a next examination question, and repeating the operation processes from the first step to the sixth step until the specified number of questions is reached (namely, simplifying the evaluation content).
By adopting the evaluation method based on the CO-MIRT algorithm model, balance among the measurement error, the measurement quantity and the number of the knowledge points to be measured can be considered, evaluation content is simplified, event evaluation efficiency is improved, high precision of an evaluation effect can be realized, and use experience of a user is effectively improved.
Corresponding to the evaluation method based on the CO-MIRT algorithm model, the invention also provides an evaluation device based on the CO-MIRT algorithm model. Since the embodiment of the device is similar to the embodiment of the method, the description is simple, and for the relevant points, please refer to the description in the embodiment of the method, and the embodiment of the evaluation device based on the CO-MIRT algorithm model described below is only illustrative. Fig. 2 is a schematic diagram of an evaluation apparatus based on a CO-MIRT algorithm model according to an embodiment of the present invention.
The invention relates to an evaluation device based on a CO-MIRT algorithm model, which comprises the following parts:
a parameter information obtaining unit 201, configured to obtain basic parameter information of the evaluation event.
A parameter information compiling unit 202, configured to compile the basic parameter information into a knowledge point hierarchy set and a test question information set.
And the setting list unit 203 is used for setting a test question ID list and a student answer list according to the knowledge point hierarchy set and the test question information set.
And the evaluation analysis unit 204 is used for performing evaluation analysis based on the input test question ID list and the student answer list by adopting a preset CO-MIRT algorithm model after the target object completes answering, and outputting a test grade analysis result.
By adopting the evaluation device based on the CO-MIRT algorithm model, balance among the measurement error, the measurement quantity and the number of the knowledge points to be measured can be considered, evaluation content is simplified, event evaluation efficiency is improved, high precision of an evaluation effect can be realized, and use experience of a user is effectively improved.
Corresponding to the evaluation method based on the CO-MIRT algorithm model, the invention also provides electronic equipment. Since the embodiment of the electronic device is similar to the above method embodiment, the description is relatively simple, and please refer to the description of the above method embodiment, and the electronic device described below is only schematic. Fig. 3 is a schematic diagram of an electronic device according to an embodiment of the present invention.
The electronic device specifically includes: a processor 301 and a memory 302; the memory 302 is configured to run one or more program instructions, and is configured to store a program of the CO-MIRT algorithm model-based evaluation method, and after the server is powered on and runs the program of the CO-MIRT algorithm model-based evaluation method through the processor 301, the CO-MIRT algorithm model-based evaluation method is executed.
Corresponding to the evaluation method based on the CO-MIRT algorithm model, the invention also provides a computer storage medium. Since the embodiment of the computer storage medium is similar to the above method embodiment, the description is simple, and please refer to the description of the above method embodiment for relevant points, and the computer storage medium described below is only schematic.
The computer storage medium contains one or more program instructions for performing the CO-MIRT algorithm model-based assessment method described above.
In an embodiment of the invention, the processor or processor module may be an integrated circuit chip having signal processing capabilities. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or may be implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. storage media well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (ddr Data Rate SDRAM), Enhanced SDRAM (ESDRAM), synchlronous DRAM (SLDRAM), and Direct memory bus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention can be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only illustrative of the present invention and are not intended to limit the scope of the present invention, and any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.