CN113569870A - Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network - Google Patents

Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network Download PDF

Info

Publication number
CN113569870A
CN113569870A CN202110876668.2A CN202110876668A CN113569870A CN 113569870 A CN113569870 A CN 113569870A CN 202110876668 A CN202110876668 A CN 202110876668A CN 113569870 A CN113569870 A CN 113569870A
Authority
CN
China
Prior art keywords
nodes
node
cross
similarity
modal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110876668.2A
Other languages
Chinese (zh)
Other versions
CN113569870B (en
Inventor
宋凌云
刘至臻
尚学群
张颖
李战怀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN202110876668.2A priority Critical patent/CN113569870B/en
Publication of CN113569870A publication Critical patent/CN113569870A/en
Application granted granted Critical
Publication of CN113569870B publication Critical patent/CN113569870B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention discloses a cross-modal problem Q matrix automatic construction method based on a heterogeneous graph neural network, which constructs a heterogeneous graph simultaneously containing cross-modal problems and knowledge points, establishes connection between the problems and the knowledge points according to similarity, provides a heterogeneous graph neural network for learning node representation of the heterogeneous graph, learns node feature representation in the heterogeneous graph and link prediction between problem nodes and knowledge point nodes, and finds out a corresponding relation between the problems and the knowledge points so as to achieve the purpose of automatically constructing a Q matrix. The invention directly searches the association between the problem and the knowledge point in the heterogeneous graph through the link prediction in the graph neural network, thereby reducing the operation burden of the computer. In addition, the similarity information among the problems is introduced into the prediction process of the knowledge points, and the relation among the problem nodes is increased, so that the accuracy of the prediction of the follow-up knowledge points is improved.

Description

Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network
Technical Field
The invention belongs to the technical field of expert systems, and particularly relates to a cross-modal problem Q matrix automatic construction method.
Background
The cognitive diagnosis is an important function of an online education system, the grasping conditions of students on relevant knowledge points can be automatically evaluated through the answering scores of the students, teachers are helped to better analyze learning deficiencies and defects of the students, the teachers are enabled to know and improve the shortcomings of teaching, the teaching and evaluation are better combined, and the students are promoted to grow more effectively. The Q matrix is formed by the incidence relation of problems and knowledge points, and is an important basis for parameter estimation and result prediction of the cognitive diagnosis model.
The existing Q matrix is often constructed manually by experts according to the problem, which needs to consume a large amount of labor cost and is difficult to realize on a large-scale data set. Therefore, the method for automatically constructing the Q matrix is a premise and a basis for applying the cognitive diagnosis model to a large-scale problem data set. The existing automatic Q matrix construction methods are generally directed to single-mode problems only containing text descriptions, and have the following defects: is not suitable for the cross-modal problem. In the cross-modal problem of image-text mixing, it is often difficult to find out all knowledge points of problem investigation comprehensively only by using text description in the problem, and text and picture information must be combined, so that the methods are difficult to be directly applied to the cross-modal problem. And secondly, neglecting the relation between the problems. Similar questions tend to explore similar knowledge points, and two questions exploring the same knowledge points tend to be similar. ③ depending on the amount of data. Many methods are based on the answer scores of students when the Q matrix is constructed, and when the answer score data of the students are less, many existing methods lose the accuracy.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a cross-modal problem Q matrix automatic construction method based on a heterogeneous graph neural network, which constructs a heterogeneous graph simultaneously containing cross-modal problems and knowledge points, establishes connection between the problems and the knowledge points according to similarity, provides a heterogeneous graph neural network for learning node representation of the heterogeneous graph, learns node feature representation in the heterogeneous graph and link prediction between problem nodes and knowledge point nodes, and finds out corresponding relations between the problems and the knowledge points, so as to achieve the purpose of automatically constructing the Q matrix. The invention directly searches the association between the problem and the knowledge point in the heterogeneous graph through the link prediction in the graph neural network, thereby reducing the operation burden of the computer. In addition, the similarity information among the problems is introduced into the prediction process of the knowledge points, and the relation among the problem nodes is increased, so that the accuracy of the prediction of the follow-up knowledge points is improved.
The technical scheme adopted by the invention for solving the technical problem comprises the following steps:
step 1: constructing a heteromorphic graph;
step 1-1: setting a node;
respectively corresponding each cross-modal problem to a cross-modal problem node, and respectively corresponding each knowledge point to a knowledge point node to obtain two kinds of nodes which are respectively a total number of N cross-modal problem nodes and a total number of M knowledge point nodes;
step 1-2: constructing a problem-problem edge;
respectively calculating text similarity and picture similarity among cross-modal problems by using BERT and perceptual hashing algorithm, and enabling the text similarity to be larger than y1And the picture similarity is greater than y2The nodes corresponding to the cross-modal problem are connected to obtain a problem-problem edge;
step 1-3: constructing problem-knowledge point edges;
judging whether the knowledge points are investigated by the cross-modal problem according to the label information of the knowledge points investigated by the cross-modal problem in the data set, and if so, connecting the nodes corresponding to the cross-modal problem with the nodes corresponding to the investigated knowledge points to obtain a problem-knowledge point edge;
step 1-4: constructing a knowledge point-knowledge point edge;
calculating text similarity between knowledge points by using BERT (belief propagation), and enabling the text similarity to be larger than y1To be aware ofConnecting the nodes corresponding to the identification points to obtain knowledge points and knowledge point edges;
step 2: constructing a Q matrix based on a heterogeneous graph neural network;
step 2-1: constructing node characteristics based on different types of neighbors;
in the heterogeneous graph obtained in the step 2, a feature matrix Z is obtained by synthesizing the neighbor features of the nodes1；
Step 2-2: constructing node characteristics based on the meta path;
with Z1For inputting, the characteristics of the problem-knowledge point-problem element path and the knowledge point-problem-knowledge point element path in the abnormal picture are integrated to obtain the node embedding Z2；
Step 2-3: constructing node characteristics based on a ring;
with Z2For input, synthesizing the characteristics of a ring structure in a heterogeneous graph to obtain a final node embedding Z;
step 2-4: link prediction based on node characteristics;
using the dot product of the node feature vectors as the score of the edge, the calculation process is as follows:
score＝hu·hv
where score represents the score of the edge, hu、hvRespectively representing feature vectors of two nodes connected by the edge;
the interval loss is used as a loss function, and the calculation process is as follows:
loss＝mean(1-scorepos+scoreneg)
wherein scoreposScore, representing the edge in the test setnegA score representing an absent edge; the score of the edge is in direct proportion to the probability of the edge between the two nodes, and the higher the score is, the higher the probability of the edge between the two nodes is; when an edge exists between the cross-modal problem node and the knowledge point node, judging that the cross-modal problem investigates the knowledge point;
further, the text similarity calculation method is as follows:
step 1-2-1: generating a vector representation for each text using BERT;
step 1-2-2: and calculating the similarity between the two texts in each group by using the cosine similarity, wherein the specific method comprises the following steps:
Figure BDA0003190540530000031
where X, Y represent two different texts,
Figure BDA0003190540530000032
vector representations of X and Y, | X |, | Y |, respectively
Figure BDA0003190540530000033
The die length of (2); the finally obtained similarity value is between 0 and 1, and the closer the obtained result is to 1, the higher the similarity is;
step 1-2-3: screening and keeping text similarity greater than y1If the group is a knowledge point, connecting the knowledge point nodes corresponding to the group to obtain a knowledge point-knowledge point edge; if the grouping is a cross-modal problem, the grouping is listed in a cross-modal problem list with similar texts to wait for the next processing;
step 1-2-4: obtaining M knowledge points between each two
Figure BDA0003190540530000034
Text similarity of individual groups; and N cross-modal problems, two by two, in total
Figure BDA0003190540530000035
The text similarity of each group.
Further, the method for calculating the image similarity is as follows:
suppose that among the N cross-modal problems, S cross-modal problems are in a cross-modal problem list with similar texts, and the S cross-modal problems are paired pairwise to obtain a common cross-modal problem
Figure BDA0003190540530000036
A group of the data; combining the pictures contained in the two cross-modal problems in pairs, calculating the similarity between the pictures by using a perceptual hash algorithm, and obtaining the picture similarity between the two cross-modal problems by using a greedy algorithm;
the specific steps for obtaining the picture similarity between the questions by using the greedy algorithm are as follows:
step 1-2-5: obtaining a picture similarity matrix corresponding to the two problems according to the picture similarity between the two problems; assuming that there are m and n pictures in the two problems, the similarity matrix is as follows:
Figure BDA0003190540530000037
wherein I1,11 st picture showing the 1 st question, and so on, I2,nThe nth picture, sim (I) representing the 2 nd question1,1,I2,1) Is represented by1,1And I2,1Similarity of (c), by analogy, sim (I)1,m,I2,n) Is represented by1,mAnd I2,nThe similarity of (2);
step 1-2-6: selecting the value with the highest similarity in the similarity matrix, recording, and deleting the row and the column corresponding to the value with the highest similarity:
step 1-2-7: repeating the steps 1-2-6 until the matrix is empty, and obtaining min (m, n) similarity values;
step 1-2-8: averaging the recorded min (m, n) similarity values to serve as the image similarity among the cross-modal problems;
after the image similarity among the cross-modal problems with similar texts is obtained, keeping the image similarity value larger than y2The cross-modal problem groups are grouped, and the corresponding cross-modal problem nodes are connected to obtain a problem-problem edge.
Further, the step 2-1 synthesizes the neighbor characteristics of the nodes to obtain a characteristic matrix Z1The method comprises the following steps:
defining the same typeThe nodes being homogeneous nodes, e.g. problem nodes q1And problem node q2(ii) a Defining nodes of different types as heterogeneous nodes, e.g. problem node q1And knowledge point node k1；
Respectively aggregating the characteristics of the homogeneous nodes by using GCN, and then aggregating the heterogeneous nodes in an attention mode to obtain new node characteristics epsilonv(ii) a The specific formula is as follows:
εv＝αvhv+αqhq+αkhk
wherein h isqAggregated representation, h, of neighbors representing the type of node in questionkAggregated representation, h, of neighbors representing all knowledge point node typesvA node feature representing an input; alpha is alpha*For the importance represented by different nodes, the calculation method is as follows:
Figure BDA0003190540530000041
where u is a learnable attention parameter,
Figure BDA0003190540530000042
d represents the length of the input feature vector, and the nodes of all the nodes are represented to form a feature matrix Z1。
Further, the specific method of step 2-2 is as follows:
when two homogeneous nodes are indirectly connected through a heterogeneous node, the path is a meta path: question-knowledge point-question and knowledge point-question-knowledge point, which respectively represent two questions that have been investigated for the same knowledge point and two knowledge points that have been investigated simultaneously by one question; feature matrix Z to be input1As the characteristics of each node in the graph, firstly, the GCN is used for aggregating the characteristics of two-hop neighbors related to the meta-path to one-hop neighbors related to the meta-path, and then the characteristics of one-hop neighbors related to the meta-path are aggregated to the node per se to obtain a characteristic matrix Z2。
Further, the specific method of step 2-3 is as follows:
finding out a triangular ring containing heterogeneous nodes in the heterogeneous graph, and converting the characteristic matrix Z2Taking the average value of the node characteristics in the triangular ring as the characteristics of the ring; if a certain node exists in a plurality of rings, the characteristics of the rings are respectively averaged according to different types, and h is usedaFeatures representing a ring containing one knowledge point node and two problem nodes, by hbFeatures representing a ring containing a problem node and two knowledge point nodes, hwIs the input of the layer; the specific calculation method is as follows:
εs＝αwhw+αaha+αbhb
wherein epsilonsIs represented by a node*For the importance represented by different nodes, the calculation method is as follows:
Figure BDA0003190540530000051
where p is a learnable attention parameter,
Figure BDA0003190540530000052
the node representations of all nodes constitute the final node embedding Z.
Further, y is1＝0.9，y1＝0.5。
The invention has the following beneficial effects:
in the process of predicting knowledge points corresponding to the cross-modal problem, compared with the traditional method, the method starts from the cross-modal problem, constructs the heterogeneous graph with the problem and the knowledge points as nodes, directly searches the association between the problem and the knowledge points in the heterogeneous graph through the link prediction in the graph neural network, and reduces the operation burden of a computer. In addition, the similarity information among the problems is introduced into the prediction process of the knowledge points, and the relation among the problem nodes is increased, so that the accuracy of the prediction of the follow-up knowledge points is improved.
Drawings
Fig. 1 is a schematic diagram of a node and its two-hop neighbors in the embodiment of the present invention.
FIG. 2 is an illustration of a ring in an anomaly map in an embodiment of the present invention.
FIG. 3 is a comparison of AUC for five models in the examples of the present invention.
FIG. 4 is a comparison of the loss values of five models in the example of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
Aiming at the defects and problems in the prior art, the invention provides an automatic Q matrix construction method aiming at the problem of image-text mixed cross-mode. The method does not depend on the answer scores of students, discovers the relation between the cross-modal question and knowledge points through the cross-modal question, considers the picture information and the text information in the cross-modal question at the same time, and adds the consideration of the similarity among the questions, so that the model has good effect on the automatic construction of the Q matrix of the cross-modal question.
A cross-modal problem Q matrix automatic construction method based on a heterogeneous graph neural network comprises the following steps:
step 1: constructing a heteromorphic graph;
step 1-1: setting a node;
respectively corresponding each cross-modal problem to a cross-modal problem node, and respectively corresponding each knowledge point to a knowledge point node to obtain two kinds of nodes which are respectively a total number of N cross-modal problem nodes and a total number of M knowledge point nodes;
step 1-2: constructing a problem-problem edge;
respectively calculating text similarity and picture similarity among cross-modal problems by using BERT and perceptual hashing algorithm, and enabling the text similarity to be larger than y1And the picture similarity is greater than y2The nodes corresponding to the cross-modal problem are connected to obtain a problem-problem edge;
step 1-3: constructing problem-knowledge point edges;
judging whether the knowledge points are investigated by the cross-modal problem according to the label information of the knowledge points investigated by the cross-modal problem in the data set, and if so, connecting the nodes corresponding to the cross-modal problem with the nodes corresponding to the investigated knowledge points to obtain a problem-knowledge point edge;
step 1-4: constructing a knowledge point-knowledge point edge;
calculating text similarity between knowledge points by using BERT (belief propagation), and enabling the text similarity to be larger than y1Connecting the nodes corresponding to the knowledge points to obtain knowledge points-knowledge point edges;
step 2: constructing a Q matrix based on a heterogeneous graph neural network;
step 2-1: constructing node characteristics based on different types of neighbors;
in the heterogeneous graph obtained in the step 2, a feature matrix Z is obtained by synthesizing the neighbor features of the nodes1；
Step 2-2: constructing node characteristics based on the meta path;
with Z1For inputting, the characteristics of the problem-knowledge point-problem element path and the knowledge point-problem-knowledge point element path in the abnormal picture are integrated to obtain the node embedding Z2；
Step 2-3: constructing node characteristics based on a ring;
with Z2For input, synthesizing the characteristics of a ring structure in a heterogeneous graph to obtain a final node embedding Z;
step 2-4: link prediction based on node characteristics;
using the dot product of the node feature vectors as the score of the edge, the calculation process is as follows:
score＝hu·hv
where score represents the score of the edge, hu、hvRespectively representing feature vectors of two nodes connected by the edge;
the interval loss is used as a loss function, and the calculation process is as follows:
loss＝mean(1-scorepos+scoreneg
wherein scoreposRepresenting test setsScore of the edge in (1), scorenegA score representing an absent edge; the score of the edge is in direct proportion to the probability of the edge between the two nodes, and the higher the score is, the higher the probability of the edge between the two nodes is; when an edge exists between the cross-modal problem node and the knowledge point node, judging that the cross-modal problem investigates the knowledge point;
further, the text similarity calculation method is as follows:
step 1-2-1: generating a vector representation for each text using BERT;
step 1-2-2: and calculating the similarity between the two texts in each group by using the cosine similarity, wherein the specific method comprises the following steps:
Figure BDA0003190540530000071
where X, Y represent two different texts,
Figure BDA0003190540530000072
vector representations of X and Y, | X |, | Y |, respectively
Figure BDA0003190540530000073
The die length of (2); the finally obtained similarity value is between 0 and 1, and the closer the obtained result is to 1, the higher the similarity is;
step 1-2-3: screening and keeping text similarity greater than y1If the group is a knowledge point, connecting the knowledge point nodes corresponding to the group to obtain a knowledge point-knowledge point edge; if the grouping is a cross-modal problem, the grouping is listed in a cross-modal problem list with similar texts to wait for the next processing;
step 1-2-4: obtaining M knowledge points between each two
Figure BDA0003190540530000074
Text similarity of individual groups; and N cross-modal problems, two by two, in total
Figure BDA0003190540530000075
The text similarity of each group.
Further, the method for calculating the image similarity is as follows:
suppose that among the N cross-modal problems, S cross-modal problems are in a cross-modal problem list with similar texts, and the S cross-modal problems are paired pairwise to obtain a common cross-modal problem
Figure BDA0003190540530000076
A group of the data; combining the pictures contained in the two cross-modal problems in pairs, calculating the similarity between the pictures by using a perceptual hash algorithm, and obtaining the picture similarity between the two cross-modal problems by using a greedy algorithm;
the specific steps for obtaining the picture similarity between the questions by using the greedy algorithm are as follows:
step 1-2-5: obtaining a picture similarity matrix corresponding to the two problems according to the picture similarity between the two problems; assuming that there are m and n pictures in the two problems, the similarity matrix is as follows:
Figure BDA0003190540530000081
wherein I1,11 st picture showing the 1 st question, and so on, I2,nThe nth picture, sim (I) representing the 2 nd question1,1,I2,1) Is represented by1,1And I2,1Similarity of (c), by analogy, sim (I)1,m,I2,n) Is represented by1,mAnd I2,nThe similarity of (2);
step 1-2-6: selecting the value with the highest similarity in the similarity matrix, recording, and deleting the row and the column corresponding to the value with the highest similarity:
step 1-2-7: repeating the steps 1-2-6 until the matrix is empty, and obtaining min (m, n) similarity values;
step 1-2-8: averaging the recorded min (m, n) similarity values to serve as the image similarity among the cross-modal problems;
after the image similarity among the cross-modal problems with similar texts is obtained, keeping the image similarity value larger than y2The cross-modal problem groups are grouped, and the corresponding cross-modal problem nodes are connected to obtain a problem-problem edge.
Further, the step 2-1 synthesizes the neighbor characteristics of the nodes to obtain a characteristic matrix Z1The method comprises the following steps:
defining nodes of the same type as homogeneous nodes, e.g. problem node q1And problem node q2(ii) a Defining nodes of different types as heterogeneous nodes, e.g. problem node q1And knowledge point node k1；
Respectively aggregating the characteristics of the homogeneous nodes by using GCN, and then aggregating the heterogeneous nodes in an attention mode to obtain new node characteristics epsilonv(ii) a The specific formula is as follows:
εv＝αvhv+αqhq+αkhk
wherein h isqAggregated representation, h, of neighbors representing the type of node in questionkAggregated representation, h, of neighbors representing all knowledge point node typesvA node feature representing an input; alpha is alpha*For the importance represented by different nodes, the calculation method is as follows:
Figure BDA0003190540530000082
where u is a learnable attention parameter,
Figure BDA0003190540530000083
d represents the length of the input feature vector, and the nodes of all the nodes are represented to form a feature matrix Z1。
Further, the specific method of step 2-2 is as follows:
when two homogeneous nodes are indirectly connected through a heterogeneous node, the path is a meta path: question-knowledge point-question and knowledge point-question-knowledge point, which respectively represent the investigationTwo problems of the same knowledge point and two knowledge points which are simultaneously investigated by one problem are solved; feature matrix Z to be input1As the characteristics of each node in the graph, firstly, the GCN is used for aggregating the characteristics of two-hop neighbors related to the meta-path to one-hop neighbors related to the meta-path, and then the characteristics of one-hop neighbors related to the meta-path are aggregated to the node per se to obtain a characteristic matrix Z2。
Further, the specific method of step 2-3 is as follows:
finding out a triangular ring containing heterogeneous nodes in the heterogeneous graph, and converting the characteristic matrix Z2Taking the average value of the node characteristics in the triangular ring as the characteristics of the ring; if a certain node exists in a plurality of rings, the characteristics of the rings are respectively averaged according to different types, and h is usedaFeatures representing a ring containing one knowledge point node and two problem nodes, by hbFeatures representing a ring containing a problem node and two knowledge point nodes, hwIs the input of the layer; the specific calculation method is as follows:
εs＝αwhw+αaha+αbhb
wherein epsilonsIs represented by a node*For the importance represented by different nodes, the calculation method is as follows:
Figure BDA0003190540530000091
where p is a learnable attention parameter,
Figure BDA0003190540530000092
the node representations of all nodes constitute the final node embedding Z.
Further, y is1＝0.9，y1＝0.5。
The specific embodiment is as follows:
the method is based on an automatic construction method of a cross-modal problem Q matrix of a heterogeneous graph neural network, and the model consists of two sub-modules: the system comprises a relation graph construction module crossing media problems and knowledge points and a link prediction module based on a heterogeneous graph neural network. The overall model schematic diagram is shown in fig. 1, and is specifically described as follows:
1. constructing a relation graph of cross-media problems and knowledge points;
in the field of education, the Q matrix refers to a binary matrix showing the relationship between questions and knowledge points, where the rows of the Q matrix represent the questions and the columns represent the knowledge points. The process of constructing the Q matrix is also a process of finding the corresponding relation between the problem and the knowledge point. If the problem and the knowledge point are respectively regarded as different nodes and the corresponding relation between the problem and the knowledge point is regarded as an edge, a different composition can be constructed. The corresponding relation between the prediction problem and the knowledge points is equivalent to the prediction of whether edges exist between the problem nodes and the knowledge points, namely the construction problem of the Q matrix is converted into the link prediction problem between heterogeneous nodes in the heterogeneous graph.
The invention connects the nodes of the same type through the similarity, and the nodes of different types are connected through the existing labeled information, namely: if the knowledge point k1And knowledge point k2Similarly, they are connected; if a problem q1And problem q2Similarly, they are connected; if knowledge points k are examined by the problem q, they are connected. Assuming that the heterogeneous graph has N cross-modal problem nodes and M knowledge point nodes, a construction method of an edge in the graph is described next.
1.1 construction of knowledge points-knowledge point edges
And using the similarity of the contents of the knowledge points as the basis for connection between the knowledge points. Therefore, the front and rear order of knowledge or the chapter where the knowledge point is located is not selected as the connection basis, because the front and rear order relation of the knowledge point is difficult to accurately mark under the condition of only the name of the knowledge point; the connection between the knowledge points is realized through the chapters where the knowledge points are located, and the clustering of the content of the knowledge points is substantially the same as the connection realized through the content similarity of the knowledge points. Under the condition of no knowledge point chapter mark, the similarity of the knowledge point content is used as a reference for whether to connect the knowledge point content, and the method is more convenient and simpler.
The similarity of the contents of the knowledge points, namely the similarity between the texts of the knowledge points. The method comprises the steps of firstly obtaining vector representation containing text semantic information through BERT, and then calculating cosine similarity of the vector representation to obtain text similarity. BERT introduces two tasks in pre-training, which are prediction of words in a sentence and prediction of the next sentence in a sentence, respectively. These two tasks, which are added in the pre-training, make the trained model more focused on understanding the semantics, by which the vector representation of the text generated contains the semantic information of the text, which is in accordance with expectations.
The text similarity calculation results of the knowledge points are shown in table 1:
table 1 text similarity of knowledge points
Figure BDA0003190540530000101
1.2 construction of problem-problem edges
Similar problems tend to explore the same knowledge points, and based on the property, connection between problem nodes is performed through semantic similarity between problems. Generally, in a cross-modal problem, the knowledge points it looks at usually exist both within the picture and the text, and not only in the text or only in the picture. That is, both the picture and the text contain semantic information of the question. Therefore, semantic similarity between two questions needs to start from both the picture similarity and the text similarity between the two questions, and simultaneously consider the semantics of the picture and the text, so as to reduce the information lost when calculating the similarity.
The method for calculating the text similarity between the questions is the same as the method for calculating the text similarity between the knowledge points; when the image similarity between the problems is calculated, firstly, the situation that the problems only contain one image is considered, at this time, a perceptual hash algorithm can be used for two images to calculate the similarity between the two images, and the perceptual hash algorithm is used for generating a corresponding fingerprint character string for each image. Comparing fingerprint character strings among different pictures, wherein the closer the fingerprint character strings are, the more similar the pictures are; when a question contains more than one picture, a picture similarity matrix is generated for the two questions first. Assuming that there are m and n pictures in the two problems, the similarity matrix is as follows:
Figure BDA0003190540530000111
and then continuously and repeatedly selecting the value with the highest similarity in the matrix, recording, deleting the corresponding row and column to obtain min (m, n) similarity values, and averaging the similarity values to obtain the image similarity among the problems when more than one image is obtained.
The text similarity and picture similarity of the questions are shown in table 2:
text similarity and picture similarity of the problems of Table 2
Numbering of problem 1 Numbering of problem 2 Text similarity Picture similarity
7 8 0.9150441 0.62849662
18 19 0.9121687 0.76767125
19 20 0.9023228 0.69690988
32 33 0.9044335 0.71089203
33 34 0.9041705 0.65331502
… … … …
1.3 question-construction of knowledge Point edges
Whether an edge exists between the problem node and the knowledge point node needs to be judged according to whether the problem considers the knowledge point. In the data set, the names of the knowledge points to be investigated corresponding to each question are already marked, and the knowledge points are directly connected through the information without other calculation.
2. Heterogeneous graph neural network based link prediction
In a heterogeneous graph, two nodes may be connected by different semantic paths, which are called meta-paths. A heterogeneous graph usually has a plurality of meta-paths, different meta-paths correspond to different semantics, and node representation under different semantics can be synthesized by using a graph neural network based on the meta-paths, so that the aim of predicting the relation between a problem and a knowledge point more accurately is fulfilled.
In the problem and knowledge point association prediction, a situation that a plurality of similar knowledge points are examined by the same problem and the same knowledge point is examined by the plurality of similar problems often occurs, which results in a high possibility that a plurality of rings connected with each other between nodes exist in the above-mentioned heteromorphic graph.
The final use of the heteromorphic neural network architecture of the present embodiment can be divided into three network layers. The first layer obtains the node embedding of the layer by aggregating the node representation of the neighbor nodes; the second layer takes the node embedding of the first layer as input, and obtains the node embedding of the layer by considering the relationship between homogeneous nodes indirectly connected through heterogeneous nodes; the third layer takes the node embedding of the second layer as input, and considers the relation of nodes in a ring which may appear in the graph to obtain the final node embedding. In the following description, Q represents a problem node type, and K represents a knowledge point node type.
2.1 node feature construction based on different types of neighbors
The heterogeneous graph comprises edges of three types including QQ, QK and KK, and subgraphs only comprising QQ edges, subgraphs only comprising QK edges and subgraphs only comprising KK edges can be obtained according to different types of the edges and nodes connected with the edges. For simplicity of description, G is used respectivelyQQ、GQK、GKKThey are represented. The three subgraphs are respectively sent to the GCN, so that the representation of the nodes in different subgraphs and after the neighbor information in the subgraph is synthesized, namely the representation of the nodes in the subgraph after the GCN aggregation can be obtained. Next, taking the problem node Q as an example, how to synthesize the node representations in the subgraphs is described:
problem node Q only has the possibility of edges of both QQ and QK types, so only G needs to be consideredQQAnd GQK. By using hvRepresenting the original node embedding; h isqThe representative node is GQQ(iii) is represented by (1) after GCN polymerization; h iskThe representative node is GQKIs represented by (1) after GCN polymerization. Since node representations in different subgraphs will contribute differently to the final node representation, attention is drawn here to the fact that these node tables are represented using the following formulasThe nodes in the layer are obtained by aggregationv：
εv＝αvhv+αqhq+αkhk
Wherein alpha is*Expressing the importance of different node representations, and the calculation method is as follows:
Figure BDA0003190540530000121
2.2 node feature construction based on Meta-Path
The nodes of the same type are homogeneous nodes, and the nodes of different types are heterogeneous nodes. In this paper, two problem nodes are homogeneous nodes, two knowledge point nodes are homogeneous nodes, and a problem node and a knowledge point node are heterogeneous nodes.
The output nodes of the layer above the layer of the network are represented as the node characteristics used by the layer, and the condition that two homogeneous nodes are indirectly connected through a heterogeneous node is considered, namely, meta-paths QKQ and KQK are considered, wherein the meta-paths respectively represent two problems which consider the same knowledge point and two knowledge points which are simultaneously considered by one problem.
Nodes directly connected to node v are called one-hop neighbors, and their set is N1(v) It is shown that nodes indirectly connected to node v via one-hop neighbors are called two-hop neighbors, and their set is N2(v) And (4) showing. In this layer, to retain as much information inside the meta-path as possible, the GCN is first used to pair the two-hop neighbor N2(v) The node representations are aggregated to obtain a temporary node representation of a one-hop neighbor; and then uses GCN to make one-hop neighbor N1(v) The temporary node representation of (a) is aggregated; and finally, averaging the obtained temporary representation of the one-hop neighbor and the input feature vector for aggregation. Taking meta path QKQ as an example, assume node q1The structure of its two-hop neighbors is shown in fig. 1, and it can be seen that node q is1One-hop neighbor N1(q1)＝{k1,k2}, node k1One-hop neighbor N1(k1)＝{q2,q3,q4}, node k2One-hop neighbor of is N1(k2)＝{q5,q6Is then q1Is N2(q1)＝{q2,q3,q4,q5,q6}. Firstly, q is put in2,q3,q4And q is5,q6Respectively to k1And k2Repolymerization of k1And k2To q1To obtain the present layer q1Is represented by the node(s).
2.3 Ring-based node feature construction
In the above-described abnormal pattern, there is a high possibility that a ring structure occurs. The ring shown in fig. 2(a) indicates that two similar questions investigate the same knowledge point, and the ring shown in fig. 2(b) indicates that one question investigates two similar knowledge points.
Finding out a ring in the heteromorphic graph as shown in FIG. 2, and taking an average value of node features in the ring as the features of the ring. If a certain node exists in a plurality of rings, the characteristics of the rings are respectively averaged according to different types, and h is usedaFeatures of the ring shown in FIG. 2(a) are shown by hbRepresenting the characteristics of the ring, h, as shown in FIG. 2(b)vIs the input of the present layer, i.e. the output of the upper layer. These node representations are aggregated using equation (3-3) to obtain the present layer output εv：
εv＝αvhv+αaha+αbhb
Wherein alpha is*Expressing the importance of different node representations, and the calculation method is as follows:
Figure BDA0003190540530000131
2.4 Link prediction
The goal of link prediction is to compute the task that the node representation is upstream of based on the known nodes in the graph and the probability that an edge exists between the two nodes predicted by the edge. After the node representations are obtained by using the neural network of the graph, the probability of edges existing between two nodes needs to be calculated through the node representations, so that the purpose of link prediction is achieved.
Using the dot product of the node feature vectors as the score of the edge, the formula is as follows:
score＝hu·hv
wherein h isu、hvThe node embedding respectively represents two nodes connected by the edge, the score of the edge is in direct proportion to the probability of the link between the two nodes, and the higher the score is, the higher the probability of the edge exists is.
3. Cross-modal problem Q matrix automatic construction method training based on heterogeneous graph neural network
3.1 acquisition and processing of data sets
The embodiment acquires questions from the primary school teaching part in the ant learning network. The website classifies the topics according to the knowledge points, each knowledge point comprises different numbers of topics, pictures and texts exist in some topics, and only texts exist in some topics. The examination questions which are common in the teaching of domestic primary schools at present are obtained from a website through a Python crawler and are classified according to subject and knowledge points. The topics and knowledge points in the data set are shown in table 3.
TABLE 3 number of knowledge points and topics in the data set
Figure BDA0003190540530000141
The research object of the embodiment is a cross-modal problem of image-text mixing, so that the problem of only text and only image needs to be deleted in advance, and the problem of having both text and image is reserved. After deleting the problems which do not meet the conditions, removing the knowledge points which are not investigated by the problems, and then corresponding the remaining problems with the investigated knowledge points.
3.2 model building and training
Randomly pick out 10% of the problem-knowledge point edges as test set, and the rest is used as training set. And (3) carrying out model building by using a DGL-Pythrch frame, wherein Adam is used as an optimizer, interval loss is used as a loss function, and the learning rate is set to be 0.05 in the experiment. The interval loss is calculated as follows:
loss＝mean(1-scorepos+scoreneg)
wherein, scoreposScore, representing the edge score between nodes with connections in the artworknegIndicating the edge scores between nodes where no connection exists.
The accuracy of the algorithm is measured by using an index AUC commonly used in link prediction, the specific calculation method of the AUC is to calculate the edge score and the non-existing edge score in a test set firstly, and then the AUC is obtained by using the following formula, wherein the higher the AUC is, the better the prediction effect is.
Figure BDA0003190540530000142
Where n represents the total number of comparisons, n' represents the number of edge scores in the test set that are greater than the number of edge scores that are not present, and n "represents the number of edge scores in the test set that are equal to the number of edge scores that are not present. It can be seen that when all the edges are scored at random, AUC is 0.5. Therefore, the greater the AUC, the higher the prediction accuracy.
3.3 comparative experiments with other common methods
Selecting four comparison methods, namely GCN, RGCN, Layer1 and Layer2, wherein GCN represents that graph convolution is carried out on subgraphs of each edge type respectively, and then aggregation is carried out by using an average value; layer1 is a node feature construction part based on different types of neighbors in the invention, and Layer2 is a node feature construction part based on meta-paths in the invention; RGCN consists of two layers of GCNs, each with node embeddings aggregated by sum function, and an activation function ReLu. The results of the experiment are shown in table 4:
table 4 results obtained for different models on two data sets
Figure BDA0003190540530000151
The data in table 4 are shown in a broken line graph form, as shown in fig. 3 and 4. It can be seen that the model proposed in this aspect has good effect on link prediction on a heterogeneous graph constructed across modal problems and knowledge points, compared to RGCN. In addition, the three parts of the model all provide some help for the final result.
The GCN is a network proposed in the starting period of a graph neural network, is designed for a homogeneous graph, and is insufficient in capturing the relation between heterogeneous nodes; the RGCN is a graph neural network designed for a heterogeneous graph, different aggregation modes are designed for different edges, but the RGCN only considers the relationship between directly adjacent nodes and neglects the influence of indirectly connected nodes and rings on the node representation if the rings exist in the graph. Layer1, the first network Layer of the neural network proposed in this document, is not as expected, but only considers the relationship between neighboring nodes, and is not as good as RGCN, probably because RGCN uses several convolution modules in this chapter, while Layer1 performs only one convolution. Layer2 acts as a second network Layer, and the reason why the results are inferior to the final model results is to ignore the effects of the neighborhood nodes and the nodes in the ring. The heterogeneous graph neural network provided by the invention is designed aiming at the cross-modal problem and the link prediction among the knowledge points, and the influence of similar problems, similar knowledge points, the investigation of the relationship among the problems of the same knowledge point and the like is considered, so that the final result achieves the expected effect.

Claims (7)

1. A cross-modal problem Q matrix automatic construction method based on a heterogeneous graph neural network is characterized by comprising the following steps:
step 1: constructing a heteromorphic graph;
step 1-1: setting a node;
respectively corresponding each cross-modal problem to a cross-modal problem node, and respectively corresponding each knowledge point to a knowledge point node to obtain two kinds of nodes which are respectively a total number of N cross-modal problem nodes and a total number of M knowledge point nodes;
step 1-2: constructing a problem-problem edge;
respectively calculating text similarity and picture similarity among cross-modal problems by using BERT and perceptual hashing algorithm, and enabling the text similarity to be larger than y1And the picture similarity is greater than y2The nodes corresponding to the cross-modal problem are connected to obtain a problem-problem edge;
step 1-3: constructing problem-knowledge point edges;
judging whether the knowledge points are investigated by the cross-modal problem according to the label information of the knowledge points investigated by the cross-modal problem in the data set, and if so, connecting the nodes corresponding to the cross-modal problem with the nodes corresponding to the investigated knowledge points to obtain a problem-knowledge point edge;
step 1-4: constructing a knowledge point-knowledge point edge;
calculating text similarity between knowledge points by using BERT (belief propagation), and enabling the text similarity to be larger than y1Connecting the nodes corresponding to the knowledge points to obtain knowledge points-knowledge point edges;
step 2: constructing a Q matrix based on a heterogeneous graph neural network;
step 2-1: constructing node characteristics based on different types of neighbors;
in the heterogeneous graph obtained in the step 2, a feature matrix Z is obtained by synthesizing the neighbor features of the nodes1；
Step 2-2: constructing node characteristics based on the meta path;
with Z1For inputting, the characteristics of the problem-knowledge point-problem element path and the knowledge point-problem-knowledge point element path in the abnormal picture are integrated to obtain the node embedding Z2；
Step 2-3: constructing node characteristics based on a ring;
with Z2For input, synthesizing the characteristics of a ring structure in a heterogeneous graph to obtain a final node embedding Z;
step 2-4: link prediction based on node characteristics;
using the dot product of the node feature vectors as the score of the edge, the calculation process is as follows:
score＝hu·hv
where score represents the score of the edge, hu、hvRespectively representing feature vectors of two nodes connected by the edge;
the interval loss is used as a loss function, and the calculation process is as follows:
loss＝mean(1-scorepos+scoreneg)
wherein scoreposScore, representing the edge in the test setnegA score representing an absent edge; the score of the edge is in direct proportion to the probability of the edge between the two nodes, and the higher the score is, the higher the probability of the edge between the two nodes is; and when an edge exists between the cross-modal problem node and the knowledge point node, judging that the cross-modal problem investigates the knowledge point.
2. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein the text similarity calculation method comprises the following steps:
step 1-2-1: generating a vector representation for each text using BERT;
step 1-2-2: and calculating the similarity between the two texts in each group by using the cosine similarity, wherein the specific method comprises the following steps:
Figure FDA0003190540520000021
where X, Y represent two different texts,
Figure FDA0003190540520000022
vector representations of X and Y, | X |, | Y |, respectively
Figure FDA0003190540520000023
The die length of (2); the finally obtained similarity value is between 0 and 1, and the closer the obtained result is to 1, the higher the similarity is;
step 1-2-3: screening andkeeping text similarity greater than y1If the group is a knowledge point, connecting the knowledge point nodes corresponding to the group to obtain a knowledge point-knowledge point edge; if the grouping is a cross-modal problem, the grouping is listed in a cross-modal problem list with similar texts to wait for the next processing;
step 1-2-4: obtaining M knowledge points between each two
Figure FDA0003190540520000024
Text similarity of individual groups; and N cross-modal problems, two by two, in total
Figure FDA0003190540520000025
The text similarity of each group.
3. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein the method for calculating the similarity of the pictures is as follows:
suppose that among the N cross-modal problems, S cross-modal problems are in a cross-modal problem list with similar texts, and the S cross-modal problems are paired pairwise to obtain a common cross-modal problem
Figure FDA0003190540520000026
A group of the data; combining the pictures contained in the two cross-modal problems in pairs, calculating the similarity between the pictures by using a perceptual hash algorithm, and obtaining the picture similarity between the two cross-modal problems by using a greedy algorithm;
the specific steps for obtaining the picture similarity between the questions by using the greedy algorithm are as follows:
step 1-2-5: obtaining a picture similarity matrix corresponding to the two problems according to the picture similarity between the two problems; assuming that there are m and n pictures in the two problems, the similarity matrix is as follows:
Figure FDA0003190540520000031
wherein I1，11 st picture showing the 1 st question, and so on, I2，nThe nth picture, sim (I) representing the 2 nd question1，1，I2，1) Is represented by1，1And I2，1Similarity of (c), by analogy, sim (I)1，m，I2，n) Is represented by1，mAnd I2，nThe similarity of (2);
step 1-2-6: selecting the value with the highest similarity in the similarity matrix, recording, and deleting the row and the column corresponding to the value with the highest similarity:
step 1-2-7: repeating the steps 1-2-6 until the matrix is empty, and obtaining min (m, n) similarity values;
step 1-2-8: averaging the recorded min (m, n) similarity values to serve as the image similarity among the cross-modal problems;
after the image similarity among the cross-modal problems with similar texts is obtained, keeping the image similarity value larger than y2The cross-modal problem groups are grouped, and the corresponding cross-modal problem nodes are connected to obtain a problem-problem edge.
4. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein the step 2-1 synthesizes neighbor features of nodes to obtain a feature matrix Z1The method comprises the following steps:
defining nodes of the same type as homogeneous nodes, e.g. problem node q1And problem node q2(ii) a Defining nodes of different types as heterogeneous nodes, e.g. problem node q1And knowledge point node k1；
Respectively aggregating the characteristics of the homogeneous nodes by using GCN, and then aggregating the heterogeneous nodes in an attention mode to obtain new node characteristics epsilonv(ii) a The specific formula is as follows:
εv＝αvhv+αqhq+αkhk
wherein h isqNeighbors representing problem node typesRepresentation after polymerization hkAggregated representation, h, of neighbors representing all knowledge point node typesvA node feature representing an input; alpha is alpha*For the importance represented by different nodes, the calculation method is as follows:
Figure FDA0003190540520000032
where u is a learnable attention parameter,
Figure FDA0003190540520000033
d represents the length of the input feature vector, and the nodes of all the nodes are represented to form a feature matrix Z1。
5. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein the specific method of the step 2-2 is as follows:
when two homogeneous nodes are indirectly connected through a heterogeneous node, the path is a meta path: question-knowledge point-question and knowledge point-question-knowledge point, which respectively represent two questions that have been investigated for the same knowledge point and two knowledge points that have been investigated simultaneously by one question; feature matrix Z to be input1As the characteristics of each node in the graph, firstly, the GCN is used for aggregating the characteristics of two-hop neighbors related to the meta-path to one-hop neighbors related to the meta-path, and then the characteristics of one-hop neighbors related to the meta-path are aggregated to the node per se to obtain a characteristic matrix Z2。
6. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein the specific method of the step 2-3 is as follows:
finding out a triangular ring containing heterogeneous nodes in the heterogeneous graph, and converting the characteristic matrix Z2Taking the average value of the node characteristics in the triangular ring as the characteristics of the ring; if a node existsWithin a plurality of such rings, the characteristics of the rings are averaged, respectively, by h, according to the typeaFeatures representing a ring containing one knowledge point node and two problem nodes, by hbFeatures representing a ring containing a problem node and two knowledge point nodes, hwIs the input of the layer; the specific calculation method is as follows:
εs＝αwhw+αaha+αbhb
wherein epsilonsIs represented by a node*For the importance represented by different nodes, the calculation method is as follows:
Figure FDA0003190540520000041
where p is a learnable attention parameter,
Figure FDA0003190540520000042
the node representations of all nodes constitute the final node embedding Z.
7. The method for automatically constructing the cross-modal problem Q matrix based on the neural network of the heterogeneous graph according to claim 1, wherein y is1＝0.9，y1＝0.5。
CN202110876668.2A 2021-07-31 2021-07-31 Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network Active CN113569870B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110876668.2A CN113569870B (en) 2021-07-31 2021-07-31 Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110876668.2A CN113569870B (en) 2021-07-31 2021-07-31 Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network

Publications (2)

Publication Number Publication Date
CN113569870A true CN113569870A (en) 2021-10-29
CN113569870B CN113569870B (en) 2022-06-07

Family

ID=78169704

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110876668.2A Active CN113569870B (en) 2021-07-31 2021-07-31 Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network

Country Status (1)

Country Link
CN (1) CN113569870B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004072143A (en) * 2002-08-01 2004-03-04 Sony Corp Encoder and encoding method, program, and recording medium
US20140279727A1 (en) * 2013-03-15 2014-09-18 William Marsh Rice University Sparse Factor Analysis for Analysis of User Content Preferences
CN110264091A (en) * 2019-06-24 2019-09-20 中国科学技术大学 Student's cognitive diagnosis method
US20190318650A1 (en) * 2018-04-11 2019-10-17 Electronics And Telecommunications Research Institute Method and apparatus for learner diagnosis using reliability of cognitive diagnostic model
CN111241243A (en) * 2020-01-13 2020-06-05 华中师范大学 Knowledge measurement-oriented test question, knowledge and capability tensor construction and labeling method
CN112241481A (en) * 2020-10-09 2021-01-19 中国人民解放军国防科技大学 Cross-modal news event classification method and system based on graph neural network
CN112508334A (en) * 2020-11-06 2021-03-16 华中师范大学 Personalized paper combining method and system integrating cognitive characteristics and test question text information
CN112861967A (en) * 2021-02-07 2021-05-28 中国电子科技集团公司电子科学研究院 Social network abnormal user detection method and device based on heterogeneous graph neural network
CN113140019A (en) * 2021-05-13 2021-07-20 电子科技大学 Method for generating text-generated image of confrontation network based on fusion compensation

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004072143A (en) * 2002-08-01 2004-03-04 Sony Corp Encoder and encoding method, program, and recording medium
US20140279727A1 (en) * 2013-03-15 2014-09-18 William Marsh Rice University Sparse Factor Analysis for Analysis of User Content Preferences
US20190318650A1 (en) * 2018-04-11 2019-10-17 Electronics And Telecommunications Research Institute Method and apparatus for learner diagnosis using reliability of cognitive diagnostic model
CN110264091A (en) * 2019-06-24 2019-09-20 中国科学技术大学 Student's cognitive diagnosis method
CN111241243A (en) * 2020-01-13 2020-06-05 华中师范大学 Knowledge measurement-oriented test question, knowledge and capability tensor construction and labeling method
CN112241481A (en) * 2020-10-09 2021-01-19 中国人民解放军国防科技大学 Cross-modal news event classification method and system based on graph neural network
CN112508334A (en) * 2020-11-06 2021-03-16 华中师范大学 Personalized paper combining method and system integrating cognitive characteristics and test question text information
CN112861967A (en) * 2021-02-07 2021-05-28 中国电子科技集团公司电子科学研究院 Social network abnormal user detection method and device based on heterogeneous graph neural network
CN113140019A (en) * 2021-05-13 2021-07-20 电子科技大学 Method for generating text-generated image of confrontation network based on fusion compensation

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
DEVENDRA SINGH CHAPLOT ET AL: "Learning Cognitive Models Using Neural Networks", 《 ARTIFICIAL INTELLIGENCE IN EDUCATION》 *
XUSHAN PENG ET AL: "Research on image feature extraction and retrieval algorithms based on convolutional neural network", 《JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION》 *
YUPEI ZHANG ET AL: "Meta-knowledge dictionary learning on 1-bit response data for student knowledge diagnosis", 《KNOWLEDGE-BASED SYSTEMS》 *
高朋: "认知诊断中测验Q矩阵标定及其在组卷中的应用", 《中国优秀硕士学位论文全文数据库》 *
黄振亚: "面向个性化学习的数据挖掘方法与应用研究", 《中国博士学位论文全文数据库》 *

Also Published As

Publication number Publication date
CN113569870B (en) 2022-06-07

Similar Documents

Publication Publication Date Title
Xu et al. A two-stage consensus method for large-scale multi-attribute group decision making with an application to earthquake shelter selection
CN112241481B (en) Cross-modal news event classification method and system based on graph neural network
CN111753101B (en) Knowledge graph representation learning method integrating entity description and type
CN112711953B (en) Text multi-label classification method and system based on attention mechanism and GCN
CN107835113A (en) Abnormal user detection method in a kind of social networks based on network mapping
CN111652202B (en) Method and system for solving video question-answer problem by improving video-language representation learning through self-adaptive space-time diagram model
US9536444B2 (en) Evaluating expert opinions in a question and answer system
CN110222771A (en) A kind of classification recognition methods of zero samples pictures
CN112613602A (en) Recommendation method and system based on knowledge-aware hypergraph neural network
CN109033433B (en) Comment data emotion classification method and system based on convolutional neural network
WO2022179384A1 (en) Social group division method and division system, and related apparatuses
WO2017143773A1 (en) Crowdsourcing learning method and device
CN113392197B (en) Question-answering reasoning method and device, storage medium and electronic equipment
CN115080761A (en) Semantic perception-based low-resource knowledge graph entity alignment method
CN115114421A (en) Question-answer model training method
Casalino et al. Incremental and adaptive fuzzy clustering for virtual learning environments data analysis
WO2021120588A1 (en) Method and apparatus for language generation, computer device, and storage medium
CN113743079A (en) Text similarity calculation method and device based on co-occurrence entity interaction graph
Li et al. Learning from weakly-labeled web videos via exploring sub-concepts
CN113569870B (en) Cross-modal problem Q matrix automatic construction method based on heterogeneous graph neural network
CN111506832B (en) Heterogeneous object completion method based on block matrix completion
CN113254864A (en) Dynamic subgraph generation method and dispute detection method based on node characteristics and reply path
CN112950414A (en) Legal text representation method based on decoupling legal elements
CN112101358A (en) Method for aligning phrase and picture region based on decoupling and intervention graph network
US11687796B2 (en) Document type-specific quality model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant