CN116402133B - Knowledge graph completion method and system based on structure aggregation graph convolutional network - Google Patents
Knowledge graph completion method and system based on structure aggregation graph convolutional network Download PDFInfo
- Publication number
- CN116402133B CN116402133B CN202310385458.2A CN202310385458A CN116402133B CN 116402133 B CN116402133 B CN 116402133B CN 202310385458 A CN202310385458 A CN 202310385458A CN 116402133 B CN116402133 B CN 116402133B
- Authority
- CN
- China
- Prior art keywords
- knowledge graph
- graph
- new
- entity
- relation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002776 aggregation Effects 0.000 title claims abstract description 67
- 238000004220 aggregation Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000006870 function Effects 0.000 claims description 73
- 239000013598 vector Substances 0.000 claims description 67
- 239000011159 matrix material Substances 0.000 claims description 65
- 238000012549 training Methods 0.000 claims description 38
- 238000004364 calculation method Methods 0.000 claims description 14
- 230000004913 activation Effects 0.000 claims description 11
- 238000005096 rolling process Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000001502 supplementing effect Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000003169 complementation method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a knowledge graph completion method and a system based on a structure aggregation graph convolutional network. And extracting the head entity-relation pair in the test set and all the entities in the test set as tail entities to obtain a new triplet set, and extracting a semantic rule set corresponding to the new triplet set by using the LSTM network again. Inputting the test set into a trained network, and evaluating the effect of map completion by using a scoring function and a scoring function. The map completion method of the invention has better map completion performance than some existing methods.
Description
Technical Field
The invention relates to the field of software, in particular to a knowledge graph completion method and system based on a structural aggregation graph convolution network.
Background
The Knowledge Graph (KG) is a form of semantic network, is a graphical knowledge base for storing semantic knowledge, and has the basic composition unit of a triplet, the expression form of (h, r, t), and the h, r and t are respectively a head entity, a relation and a tail entity. And establishing connection among the entities through the relationship to form a knowledge base of the network structure. Although the knowledge graph contains rich knowledge, the knowledge contained in the knowledge graph may be incomplete, and the hidden relations of which the part is not found are missing. The purpose of map completion is to predict missing entities or relationships and perfect the knowledge triples of the knowledge map as much as possible. The current main knowledge graph completion method can be divided into a distance model, a bilinear model, a nerve tensor model, a matrix decomposition model and a translation model, wherein the distance model, the translation model and the like convert entities and relations from words into vector representations in continuous low-dimensional space, which is called as embedding of the knowledge graph. After embedding the atlas, the embedded vector is used to predict potential knowledge triples, such as TransH, transR, complEx models. The Graph Convolution Network (GCN) is a convolution neural network which can directly act on a graph and utilize the structural information of the graph, the convolution process is to sum the node information and the domain node information to obtain new node representation, the information of multiple layers of domain nodes can be fused through multi-layer convolution operation, and the node information participating in operation is more abundant. The combination of the relation graph rolling network (R-GCN) and DistMult models on the knowledge graph public data set with the graph structure is used for the graph completion task, so that the effectiveness of the GCN in the graph completion task is proved to a certain extent.
However, when the existing GCN is applied to map completion, the aggregation operation only uses the information of the central node and the nodes in the field of the GCN, and the important relation vector information existing between the nodes of the knowledge map is ignored, so that the completion effect of the GCN is poor.
Disclosure of Invention
The invention aims to provide a knowledge graph completion method and system based on a structural aggregation graph convolution network, which are used for solving the problems in the background technology.
In order to achieve the above purpose, the present invention provides the following technical solutions:
A knowledge graph completion method based on a structural aggregation graph convolution network comprises the following steps:
Step S201, obtaining a to-be-complemented knowledge graph, wherein the to-be-complemented knowledge graph consists of a plurality of triples, and each triplet consists of an entity-relation-entity;
Step S202, inputting a knowledge graph to be complemented into an LSTM network, and learning semantic connection rules of triples in the knowledge graph by the LSTM network to obtain a first semantic rule feature set S of the knowledge graph to be complemented;
step S203, inputting the knowledge graph to be complemented into a Word2Vec model, and converting the knowledge graph from Word representation to vector representation of a low-dimensional continuous space to obtain a preliminary embedded representation result;
step S204, inputting the primary embedding representation result into a structure aggregation convolution network to perform deep embedding, wherein the deep embedding process comprises the following steps: constructing a training set which is a preliminarily embedded knowledge graph containing correct labels; inputting the training set into a structure aggregation graph convolution network, training the structure aggregation convolution network, wherein the structure aggregation convolution network comprises L layers, each layer contains a trainable parameter omega, and the loss function is a two-class cross entropy loss function:
Step S2041, extracting rows of the corresponding tag matrix of the head entity-relation pairs in the training set triples according to batches, wherein all entities in the training set correspond to columns of the tag matrix to obtain a tag matrix with a size of batch_num_ent, the value of batch is the number of the head entity-relation pairs in the training batch, and the value of num_ent is the number of the training set entities; the element values of the tag matrix are valued according to the triplets in the training set, the triplets are split into head entity-relation pairs and tail entities, the values in the rows and columns corresponding to the tag matrix are 1, and the tag matrix does not have the head entity-relation pairs, and the values of the elements corresponding to the tail entities are 0; simultaneously obtaining a candidate new triplet set composed of head entity-relation pairs and all entities in the training set;
Step S2042, inputting semantic representations corresponding to the new triplet sets into an LSTM network to extract semantic features, and obtaining a second semantic rule feature set S new;
Step S2043, inputting the new triplet set into a scoring function f (h, r, t), a first semantic rule feature set S and a second semantic rule feature set S new into a scoring function g (S new, S), summing the result of the scoring function f (h, r, t) and the result of the scoring function g (S new, S), and inputting a Sigmoid function to evaluate the score of each new triplet in the new triplet candidate set, wherein the value range of the score is [0,1]; inputting BCELoss the obtained scores and a real triplet tag matrix of the training set into a loss function to calculate errors, carrying out back propagation according to the errors to update training parameters, and training until BCELoss the loss function is smaller than a preset threshold value to obtain a trained structural aggregation convolutional network;
Step S205, inputting the knowledge graph to be complemented into a trained structural aggregation convolutional network, randomly combining head entity-relation pairs, taking entities in the knowledge graph to be complemented as tail entities in sequence to form a candidate triplet set, and calculating the final scores of triples in the set; and calculating each candidate triplet set based on the head entity-relation pair, taking the triplet with the highest final score in each candidate triplet set to form a complete triplet set, removing the original triplet in the to-be-completed knowledge graph in the complete triplet set to obtain the final complete triplet set, and adding the correct triplet in the final complete triplet set into the to-be-completed knowledge graph after manual verification to realize completion of the graph.
Further improvements, the LSTM network comprises an input x t, an output gate o t, an hidden layer state h t, a forget gate f t, an input gate i t, a cell state C t, a temporary cell stateThe final result is obtained through the cyclic calculation of the memory of useful information and the forgetting of useless information;
the forgetting gate is calculated as follows:
ft＝σ(Wf·[ht-1,xt]+bf) (1)
Input gate calculation:
it＝σ(Wi·[ht-1,xt]+bi) (2)
New cell state calculation:
Output gate calculation:
ot＝σ(Wo·[ht-1,xt]+bo) (5)
ht＝ot*tanh(Ct) (6)
Obtaining a semantic feature set S through LSTM network learning;
Wherein f t、it、ot is the calculation result of forgetting gate, input gate and output gate respectively, sigma is the activation function, h t-1、ht is the hidden layer state of the previous time and the current time respectively, W f、Wi、Wo、Wc is the weight parameters of forgetting gate, input gate, output gate and new unit state for the current input x t and the hidden layer state output h t-1 of the previous unit respectively, b f、bi、b0、bc is the bias vector corresponding to the corresponding weight parameters respectively, tanh is the hyperbolic tangent function, C t-1、Ct, The state of the unit at the previous moment, the state of the unit at the current moment and the state of the temporary unit at the current moment are respectively defined.
Further improving, if the graph nodes of the knowledge graph to be complemented are n and the embedding dimension is D, inputting the GCN comprises an adjacent matrix A of a node characteristic matrix H, n of n and a degree matrix D of n; the aggregation graph rolling network performs structure aggregation operation on entity nodes:
Wherein i is the sequence number of the map entity node, j is the sequence number of one adjacent node in all adjacent nodes of the ith entity node; n i l is the result of the vector representation of the center node after the first layer convolution aggregation operation, v j is all the neighboring nodes of the center node N i, N j l-1 is the vector representation of the neighboring nodes, R j is the relationship vector between the center node N i and the neighboring node N j, For the weight parameter of the corresponding relation vector R j, ω ii is the weight parameter of the central node N i to the self value, ω ij is the weight parameter of the central node N i to the adjacent node N j to the c is the identification variable, the relation takes 1 for the adjacent node N j and takes 1 for the input degree; deg i is the sum of the outgoing and incoming degrees of the central node, and deg j is the sum of the outgoing and incoming degrees of the adjacent nodes;
In the structure aggregation convolution network, the information of the node part and the relation part has different importance degrees, and the weighted summation is carried out, wherein the matrix expression is as follows:
Wherein f (H (l), A) is a node information aggregation function of the first layer convolution network, and sigma is an activation function; h (l) is a vector representation matrix of entity nodes, the size is n×d, n is the number of entity nodes, and d is the dimension of the embedded vector; a is an adjacent matrix of entity nodes in the map, the value of the connection relation between two entities is 1, otherwise, the value is 0, and the size is n; r is a relation matrix of m x d, m is the number of relations, d is the dimension of a relation embedding vector and the embedding dimension of an entity node are the same; m is a matrix of n x M, each row corresponds to an entity node i, each column corresponds to a relation j, and the element value of the matrix represents the number of the relation j contained in the entity i; obtained by A+I, I is an identity matrix, matrix/> Compared with the matrix A, the self-connection relation of the entity nodes is increased, and the information omission of the entity nodes is avoided when aggregation operation is carried out; /(I)The diagonal element value is the summation result of the row vector elements corresponding to the matrix A; w (l),/>And the weight parameters of the information aggregation of the entity part and the relation part are d.
Further, in the step S207, the scoring function is:
f(h,r,t)＝similarity(h,r)+similarity(h,t)+similarity(r,t) (10)
Wherein f (h, r, t) is a scoring function, h is a head entity embedding vector, r is a relation embedding vector, and t is a tail entity embedding vector;
The semantic feature set S obtained by the semantic feature learning module is calculated in a scoring function, and whether the new triplet meets the scoring is determined, wherein the scoring function is as follows:
Wherein g (S new, S) is a division function, mathematical symbol Indicating the presence; s new is a feature vector in the second semantic feature set S new, S ij is a j-th bit value of the feature vector S i in the second semantic feature set S new, and S newj is a j-th bit value of the feature vector S new; n is the length of the feature vector, and alpha and xi are respectively added value and added threshold value, which are manually set;
Calculating Euclidean distance between the feature vector S new and the vector in the first semantic rule feature set S, and conforming to the adding score if a certain result is smaller than a preset value xi; the final score is obtained by the score result and the score adding result through an activation function; the final score is obtained by the score result and the score adding result through an activation function, and is as follows:
score＝sigmoid(f(h,r,t)+g(snew,S)) (13)。
The system is used for realizing the knowledge graph completion method based on the structure aggregation graph convolutional network, and comprises a word embedding module, a semantic feature learning module, a coding module and a scoring module;
the word embedding module is used for converting the knowledge graph word representation into a vector representation of a low-dimensional continuous space;
the semantic feature learning module is used for extracting a semantic rule feature set in the knowledge graph triplet;
the coding module is a structure aggregation diagram convolution network and is used for obtaining vector representation with structural characteristics;
The scoring module consists of a scoring function and a scoring function, the scoring function does not participate in the triad scoring during iterative training, and the scoring function jointly score a new triad during map population, so as to evaluate whether the new triad is established.
Further improved, the Word embedding module is a Word2Vec model, the knowledge graph to be complemented is input into the Word2Vec model, the knowledge graph is converted into vector representation of a low-dimensional continuous space from Word representation, and a preliminary embedding representation result is obtained.
Further improvement, the semantic feature learning module is an LSTM network; inputting the knowledge graph to be complemented into an LSTM network, and entering the LSTM network to learn the semantic connection rule of the triples in the knowledge graph to be complemented to obtain the semantic rule feature set S of the knowledge graph to be complemented.
Compared with the prior art, the invention has the beneficial effects that:
The invention takes the important relation vector information existing among the knowledge graph nodes into consideration, enriches the node vector representation information, improves the link prediction performance, has stronger information aggregation performance than the existing GCN, and is beneficial to improving the effect of GCN on graph completion.
Drawings
FIG. 1 is a diagram of an LSTM network architecture;
FIG. 2 is a schematic diagram of a structural aggregate graph convolutional network of the present invention;
FIG. 3 is a schematic diagram of a graph completion method based on a structured aggregation graph rolling network according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Long-short term memory network (LSTM) is one type of recurrent neural network. The LSTM model comprises an input x t at time t, an output gate o t, an hidden layer state h t, a forgetting gate f t, a memory gate i t, a cell state C t, and a temporary cell stateThe final result is obtained through cyclic calculation through memorizing useful information and forgetting useless information, and the specific structure is shown in figure 1.
The forgetting gate is calculated as follows:
ft＝σ(Wf·[ht-1,xt]+bf) (1)
Input gate calculation:
it＝σ(Wi·[ht-1,xt]+bi) (2)
Output gate calculation:
ot＝σ(Wo·[ht-1,xt]+bo) (4)
ht＝ot*tanh(Ct) (5)
And obtaining a semantic feature set S through LSTM network learning.
When the graph rolling network (GCN) performs aggregation operation on a certain entity node, the feature vectors of the self node and the neighbor node are obtained and multiplied by corresponding parameter weights respectively to be summed, and a new node vector representation is obtained after an activation function. The graph node vector performs the aggregation operation as follows:
in the above formula, i is the sequence number of the map entity node, and j is the sequence number of a certain adjacent node in all adjacent nodes of the ith entity node; n i l is the result of the vector representation of the center node after the first layer convolution aggregation operation, v j is all the neighboring nodes of the center node N i, N j l-1 is the vector representation of the neighboring nodes, R j is the relationship vector between the center node N i and the neighboring node N j, For the weight parameter of the corresponding relation vector R j, ω ii is the weight parameter of the central node N i to the self value, ω ij is the weight parameter of the central node N i to the adjacent node N j to the c is the identification variable, the relation takes 1 for the adjacent node N j and takes 1 for the input degree; deg i is the sum of the outgoing and incoming degrees of the central node, and deg j is the sum of the outgoing and incoming degrees of the adjacent nodes.
In GCN networks with network depth L, there is a corresponding parameter matrix ω for each layer of network, which functions to give different weights to the inputs. If the map node is n and the embedding dimension is D, the input of the GCN includes a node feature matrix H, n ×n adjacent matrix a and a degree matrix D of n×n. The matrix expression corresponding to the GCN aggregation operation is as follows:
Wherein f (H (l), A) is a node information aggregation function of the first layer convolution network, and sigma is an activation function; h (l) is a vector representation matrix of entity nodes, the size is n×d, n is the number of entity nodes, and d is the dimension of the embedded vector; a is an adjacent matrix of entity nodes in the map, the value of the connection relation between two entities is 1, otherwise, the value is 0, and the size is n; r is a relation matrix of m x d, m is the number of relations, d is the dimension of a relation embedding vector and the embedding dimension of an entity node are the same; m is a matrix of n x M, each row corresponds to an entity node i, each column corresponds to a relation j, and the element value of the matrix represents the number of the relation j contained in the entity i; obtained by A+I, I is an identity matrix, matrix/> Compared with the matrix A, the self-connection relation of the entity nodes is increased, and the information omission of the entity nodes is avoided when aggregation operation is carried out; /(I)The diagonal element value is the summation result of the row vector elements corresponding to the matrix A; w (l),/>And the weight parameters of the information aggregation of the entity part and the relation part are d.
When the GCN is applied to map completion, the aggregation operation only uses the information of the central node and the domain nodes thereof, and the important relation vector information existing between the knowledge map nodes is ignored. In order to enrich node vector representation information and improve link prediction performance, aggregation operation based on GCN is proposed to use a central node to perform structure aggregation operation:
In the above formula, R j is the relation vector between the central node and the domain node N j, The relation is a weight parameter of the relation, c is an identification variable, the relation takes 1 for the out degree and takes-1 for the in degree of the field node N j. In the structure aggregation, the information of the view node part and the relation part have different importance degrees, and are weighted and summed. The matrix expression is as follows:
Wherein R is a relation matrix of m x d, m is the number of relations; m is a matrix of n x M, which is the sum of the identification variables corresponding to each relation contained in the entity node, namely
The main function of GCN in the map link prediction task is to embed the map as a whole into the representation, and after the embedding is completed, a scoring function is needed to be accessed to conduct link prediction, for example, the GCN+ DistMult model is used for completing link prediction. In order to fully utilize the graph embedded representation result obtained by the structure aggregation graph convolution network, the invention provides a method for carrying out a link prediction task by using the cosine similarity of the sum of a head entity and a relation vector and a tail entity as a scoring function, wherein the scoring function is as follows:
The semantic feature set S obtained by the semantic feature learning module is calculated in a scoring function, and whether the new triplet meets the scoring is determined, wherein the scoring function is as follows:
I.e. the new feature vector S new calculates the euclidean distance with the vectors in the set S, if a certain result is smaller than the preset value ζ, the sum is met. The final score is obtained by the score result and the score adding result through an activation function, and is as follows:
score＝sigmoid(f(h,r,t)+g(snew,S)) (12)。
According to the graph completion method based on the structure aggregation graph rolling network, a knowledge graph is initially embedded through a Word2Vec model, the initial embedding result is encoded by the structure aggregation graph rolling network to achieve deeper embedding, all new triples meeting requirements are found in the embedding result by using a scoring function and a scoring function, wherein the scoring function is the proposed f (h, r, t) cosine similarity calculation method, the scoring function is a semantic rule discrimination function, and finally the new triples are added into the original graph to complete graph completion. The atlas completion system based on the structure aggregation chart convolution network consists of a word embedding module, a semantic feature learning module, a coding module and a scoring module.
The specific implementation comprises the following steps:
Step S201, obtaining a to-be-complemented knowledge graph, wherein the to-be-complemented knowledge graph consists of a plurality of triples, and each triplet consists of an entity-relation-entity;
Step S202, inputting a knowledge graph to be complemented into an LSTM network, and learning semantic connection rules of triples in the knowledge graph by the LSTM network to obtain a first semantic rule feature set S of the knowledge graph to be complemented;
step S203, inputting the knowledge graph to be complemented into a Word2Vec model, and converting the knowledge graph from Word representation to vector representation of a low-dimensional continuous space to obtain a preliminary embedded representation result;
step S204, inputting the primary embedding representation result into a structure aggregation convolution network to perform deep embedding, wherein the deep embedding process comprises the following steps: constructing a training set which is a preliminarily embedded knowledge graph containing correct labels; inputting the training set into a structure aggregation graph convolution network, training the structure aggregation convolution network, wherein the structure aggregation convolution network comprises L layers, each layer contains a trainable parameter omega, and the loss function is a two-class cross entropy loss function:
Step S2041, extracting rows of the corresponding tag matrix of the head entity-relation pairs in the training set triples according to batches, wherein all entities in the training set correspond to columns of the tag matrix to obtain a tag matrix with a size of batch_num_ent, the value of batch is the number of the head entity-relation pairs in the training batch, and the value of num_ent is the number of the training set entities; the element values of the tag matrix are valued according to the triplets in the training set, the triplets are split into head entity-relation pairs and tail entities, the values in the rows and columns corresponding to the tag matrix are 1, and the tag matrix does not have the head entity-relation pairs, and the values of the elements corresponding to the tail entities are 0; simultaneously obtaining a candidate new triplet set composed of head entity-relation pairs and all entities in the training set;
Step S2042, inputting semantic representations corresponding to the new triplet sets into an LSTM network to extract semantic features, and obtaining a second semantic rule feature set S new;
Step S2043, inputting the new triplet set into a scoring function f (h, r, t), a first semantic rule feature set S and a second semantic rule feature set S new into a scoring function g (S new, S), summing the result of the scoring function f (h, r, t) and the result of the scoring function g (S new, S), and inputting a Sigmoid function to evaluate the score of each new triplet in the new triplet candidate set, wherein the value range of the score is [0,1]; inputting BCELoss the obtained scores and a real triplet tag matrix of the training set into a loss function to calculate errors, carrying out back propagation according to the errors to update training parameters, and training until BCELoss the loss function is smaller than a preset threshold value to obtain a trained structural aggregation convolutional network;
Step S205, inputting the knowledge graph to be complemented into a trained structural aggregation convolutional network, randomly combining head entity-relation pairs, taking entities in the knowledge graph to be complemented as tail entities in sequence to form a candidate triplet set, and calculating the final scores of triples in the set; and calculating each candidate triplet set based on the head entity-relation pair, taking the triplet with the highest final score in each candidate triplet set to form a complete triplet set, removing the original triplet in the to-be-completed knowledge graph in the complete triplet set to obtain the final complete triplet set, and adding the correct triplet in the final complete triplet set into the to-be-completed knowledge graph after manual verification to realize completion of the graph.
The comparison results of the graph rolling network (Structure Aggregation Graph Convolutional Network, SAGCN) complementation method based on structure aggregation and the existing complementation methods on the data sets FB15K-237 and YAGO3-10 are as follows:
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (6)
1. The knowledge graph completion method based on the structure aggregation graph convolution network is characterized by comprising the following steps of:
Step S201, obtaining a to-be-complemented knowledge graph, wherein the to-be-complemented knowledge graph consists of a plurality of triples, and each triplet consists of an entity-relation-entity;
Step S202, inputting a knowledge graph to be complemented into an LSTM network, and learning semantic connection rules of triples in the knowledge graph by the LSTM network to obtain a first semantic rule feature set S of the knowledge graph to be complemented;
step S203, inputting the knowledge graph to be complemented into a Word2Vec model, and converting the knowledge graph from Word representation to vector representation of a low-dimensional continuous space to obtain a preliminary embedded representation result;
step S204, inputting the primary embedding representation result into a structure aggregation convolution network to perform deep embedding, wherein the deep embedding process comprises the following steps: constructing a training set which is a preliminarily embedded knowledge graph containing correct labels; inputting the training set into a structure aggregation graph convolution network, training the structure aggregation convolution network, wherein the structure aggregation convolution network comprises L layers, each layer contains a trainable parameter omega, and the loss function is a two-class cross entropy loss function:
Step S2041: if the graph nodes of the knowledge graph to be complemented are n and the embedding dimension is D, the input of the aggregation graph rolling network comprises an adjacent matrix A of a node characteristic matrix H, n n of n and a degree matrix D of n; the aggregation graph rolling network performs structure aggregation operation on entity nodes:
Wherein i is the sequence number of the map entity node, j is the sequence number of one adjacent node in all adjacent nodes of the ith entity node; n i l is the result of the vector representation of the center node after the first layer convolution aggregation operation, v j is all the neighboring nodes of the center node N i, N j l-1 is the vector representation of the neighboring nodes, R j is the relationship vector between the center node N i and the neighboring node N j, For the weight parameter of the corresponding relation vector R j, ω ii is the weight parameter of the central node N i to the self value, ω ij is the weight parameter of the central node N i to the adjacent node N j to the c is the identification variable, the relation takes 1 for the adjacent node N j and takes 1 for the input degree; deg i is the sum of the outgoing and incoming degrees of the central node, and deg j is the sum of the outgoing and incoming degrees of the adjacent nodes;
In the structure aggregation convolution network, the information of the node part and the relation part has different importance degrees, and the weighted summation is carried out, wherein the matrix expression is as follows:
Wherein f (H (l), A) is a node information aggregation function of the first layer convolution network, and sigma is an activation function; h (l) is a vector representation matrix of entity nodes, the size is n×d, n is the number of entity nodes, and d is the dimension of the embedded vector; a is an adjacent matrix of entity nodes in the map, the value of the connection relation between two entities is 1, otherwise, the value is 0, and the size is n; r is a relation matrix of m x d, m is the number of relations, d is the dimension of a relation embedding vector and the embedding dimension of an entity node are the same; m is a matrix of n x M, each row corresponds to an entity node i, each column corresponds to a relation j, and the element value of the matrix represents the number of the relation j contained in the entity i; obtained by A+I, I is an identity matrix, matrix/> Compared with the matrix A, the self-connection relation of the entity nodes is increased, and the information omission of the entity nodes is avoided when aggregation operation is carried out; /(I)The diagonal element value is the summation result of the row vector elements corresponding to the matrix A; w (l),/>The weight parameters of the information aggregation of the entity part and the relation part are d;
Step S2042, extracting rows of the corresponding tag matrix of the head entity-relation pairs in the training set triples according to batches, wherein all entities in the training set correspond to columns of the tag matrix to obtain a tag matrix with a size of batch_num_ent, the value of batch is the number of the head entity-relation pairs in the training batch, and the value of num_ent is the number of the training set entities; the element values of the tag matrix are valued according to the triplets in the training set, the triplets are split into head entity-relation pairs and tail entities, the values in the rows and columns corresponding to the tag matrix are 1, and the tag matrix does not have the head entity-relation pairs, and the values of the elements corresponding to the tail entities are 0; simultaneously obtaining a candidate new triplet set composed of head entity-relation pairs and all entities in the training set;
Step S2043, inputting semantic representations corresponding to the new triplet sets into an LSTM network to extract semantic features, and obtaining a second semantic rule feature set S new;
Step S2044, inputting the new triplet set into a scoring function f (h, r, t), a first semantic rule feature set S and a second semantic rule feature set S new into a scoring function g (S new, S), summing the result of the scoring function f (h, r, t) and the result of the scoring function g (S new, S), and inputting a Sigmoid function to evaluate the score of each new triplet in the new triplet candidate set, wherein the value range of the score is [0,1]; inputting BCELoss the obtained scores and a real triplet tag matrix of the training set into a loss function to calculate errors, carrying out back propagation according to the errors to update training parameters, and training until BCELoss the loss function is smaller than a preset threshold value to obtain a trained structural aggregation convolutional network;
Step S205, inputting the knowledge graph to be complemented into a trained structural aggregation convolutional network, randomly combining head entity-relation pairs, taking entities in the knowledge graph to be complemented as tail entities in sequence to form a candidate triplet set, and calculating the final scores of triples in the set; and calculating each candidate triplet set based on the head entity-relation pair, taking the triplet with the highest final score in each candidate triplet set to form a complete triplet set, removing the original triplet in the to-be-completed knowledge graph in the complete triplet set to obtain the final complete triplet set, and adding the correct triplet in the final complete triplet set into the to-be-completed knowledge graph after manual verification to realize completion of the graph.
2. The method for supplementing a knowledge graph based on a structured aggregation graph convolutional network as recited in claim 1, wherein said LSTM network comprises an input x t, an output gate o t, an hidden layer state h t, a forget gate f t, an input gate i t, a cell state C t, a temporary cell stateThe final result is obtained through the cyclic calculation of the memory of useful information and the forgetting of useless information;
the forgetting gate is calculated as follows:
ft＝σ(Wfg[ht-1,xt]+bf) (1)
Input gate calculation:
it＝σ(Wig[ht-1,xt]+bi) (2)
New cell state calculation:
Output gate calculation:
ot＝σ(Wog[ht-1,xt]+bo) (5)
ht＝ot*tanh(Ct) (6)
Obtaining a semantic feature set S through LSTM network learning;
Wherein f t、it、ot is the calculation result of forgetting gate, input gate and output gate respectively, sigma is the activation function, h t-1、ht is the hidden layer state of the previous time and the current time respectively, W f、Wi、Wo、Wc is the weight parameters of forgetting gate, input gate, output gate and new unit state for the current input x t and the hidden layer state output h t-1 of the previous unit respectively, b f、bi、b0、bc is the bias vector corresponding to the corresponding weight parameters respectively, tanh is the hyperbolic tangent function, C t-1、Ct, The state of the unit at the previous moment, the state of the unit at the current moment and the state of the temporary unit at the current moment are respectively defined.
3. The knowledge graph completion method based on the structural aggregate graph convolutional network as recited in claim 1, wherein in the step S2044, the scoring function is:
f(h,r,t)＝similarity(h,r)+similarity(h,t)+similarity(r,t) (10)
Wherein f (h, r, t) is a scoring function, h is a head entity embedding vector, r is a relation embedding vector, and t is a tail entity embedding vector;
The semantic feature set S obtained by the semantic feature learning module is calculated in a scoring function, and whether the new triplet meets the scoring is determined, wherein the scoring function is as follows:
Wherein g (S new, S) is a division function, mathematical symbol Indicating the presence; s new is a feature vector in the second semantic feature set S new, S ij is a j-th bit value of the feature vector S i in the second semantic feature set S new, and S newj is a j-th bit value of the feature vector S new; n is the length of the feature vector, and alpha and xi are respectively added value and added threshold value, which are manually set;
Calculating Euclidean distance between the feature vector S new and the vector in the first semantic rule feature set S, and conforming to the adding score if a certain result is smaller than a preset value xi; the final score is obtained by the score result and the score adding result through an activation function; the final score is obtained by the score result and the score adding result through an activation function, and is as follows:
score＝sigmoid(f(h,r,t)+g(snew,S)) (13)。
4. a knowledge graph completion system based on a structure aggregation graph convolution network, which is characterized in that the system is used for realizing the knowledge graph completion method based on the structure aggregation graph convolution network according to any one of claims 1-3, and comprises a word embedding module, a semantic feature learning module, a coding module and a scoring module;
the word embedding module is used for converting the knowledge graph word representation into a vector representation of a low-dimensional continuous space;
the semantic feature learning module is used for extracting a semantic rule feature set in the knowledge graph triplet;
the coding module is a structure aggregation diagram convolution network and is used for obtaining vector representation with structural characteristics;
The scoring module consists of a scoring function and a scoring function, the scoring function does not participate in the triad scoring during iterative training, and the scoring function jointly score a new triad during map population, so as to evaluate whether the new triad is established.
5. The knowledge graph completion system based on the structure aggregation graph convolution network according to claim 4, wherein the Word embedding module is a Word2Vec model, the knowledge graph to be completed is input into the Word2Vec model, the knowledge graph is converted from Word representation into vector representation of a low-dimensional continuous space, and a preliminary embedding representation result is obtained.
6. The knowledge-graph completion system based on a structured aggregate graph convolutional network of claim 4, wherein the semantic feature learning module is an LSTM network; inputting the knowledge graph to be complemented into an LSTM network, and entering the LSTM network to learn the semantic connection rule of the triples in the knowledge graph to be complemented to obtain the semantic rule feature set S of the knowledge graph to be complemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310385458.2A CN116402133B (en) | 2023-04-12 | 2023-04-12 | Knowledge graph completion method and system based on structure aggregation graph convolutional network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310385458.2A CN116402133B (en) | 2023-04-12 | 2023-04-12 | Knowledge graph completion method and system based on structure aggregation graph convolutional network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116402133A CN116402133A (en) | 2023-07-07 |
| CN116402133B true CN116402133B (en) | 2024-04-30 |
Family
ID=87010007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310385458.2A Active CN116402133B (en) | 2023-04-12 | 2023-04-12 | Knowledge graph completion method and system based on structure aggregation graph convolutional network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116402133B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117453923B (en) * | 2023-08-30 | 2024-03-19 | 广东电白建设集团有限公司 | Method for optimizing relation between construction site construction equipment and building facilities |
| CN117273129B (en) * | 2023-10-11 | 2024-04-05 | 上海峻思寰宇数据科技有限公司 | Behavior pattern creation and generation method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347847A (en) * | 2019-07-22 | 2019-10-18 | 西南交通大学 | Knowledge mapping complementing method neural network based |
| CN112818690A (en) * | 2021-01-22 | 2021-05-18 | 润联软件系统(深圳)有限公司 | Semantic recognition method and device combined with knowledge graph entity information and related equipment |
| CN114444694A (en) * | 2022-01-21 | 2022-05-06 | 重庆邮电大学 | Open world knowledge graph complementing method and device |
| CN114443862A (en) * | 2022-01-28 | 2022-05-06 | 齐鲁工业大学 | Knowledge graph completion method and system based on weighted graph convolution network |
| CN114610900A (en) * | 2022-03-14 | 2022-06-10 | 上海交通大学 | Knowledge graph complementing method and system |
| CN115879547A (en) * | 2022-12-14 | 2023-03-31 | 山东省计算中心（国家超级计算济南中心） | Open world knowledge graph complementing method and system based on LSTM and attention mechanism |
-
2023
- 2023-04-12 CN CN202310385458.2A patent/CN116402133B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110347847A (en) * | 2019-07-22 | 2019-10-18 | 西南交通大学 | Knowledge mapping complementing method neural network based |
| CN112818690A (en) * | 2021-01-22 | 2021-05-18 | 润联软件系统(深圳)有限公司 | Semantic recognition method and device combined with knowledge graph entity information and related equipment |
| CN114444694A (en) * | 2022-01-21 | 2022-05-06 | 重庆邮电大学 | Open world knowledge graph complementing method and device |
| CN114443862A (en) * | 2022-01-28 | 2022-05-06 | 齐鲁工业大学 | Knowledge graph completion method and system based on weighted graph convolution network |
| CN114610900A (en) * | 2022-03-14 | 2022-06-10 | 上海交通大学 | Knowledge graph complementing method and system |
| CN115879547A (en) * | 2022-12-14 | 2023-03-31 | 山东省计算中心（国家超级计算济南中心） | Open world knowledge graph complementing method and system based on LSTM and attention mechanism |
Non-Patent Citations (2)
| Title |
|---|
| GRL: Knowledge graph completion with GAN-based reinforcement learning;Qi Wang等;《Knowledge-Based Systems》;20201231;论文第1-8页 * |
| 利用图神经网络进行的知识图谱补全综述;球球offer;《https://blog.csdn.net/qq_37492509/article/details/111415443》;20201220;论文第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116402133A (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116402133B (en) | Knowledge graph completion method and system based on structure aggregation graph convolutional network | |
| CN110347847B (en) | Knowledge graph complementing method based on neural network | |
| CN111291836B (en) | Method for generating student network model | |
| CN108133038B (en) | Entity level emotion classification system and method based on dynamic memory network | |
| CN111274800B (en) | Inference type reading understanding method based on relational graph convolution network | |
| CN113190688A (en) | Complex network link prediction method and system based on logical reasoning and graph convolution | |
| CN113628059B (en) | Associated user identification method and device based on multi-layer diagram attention network | |
| CN113610235B (en) | Adaptive learning support device and method based on depth knowledge tracking | |
| CN113344053B (en) | Knowledge tracking method based on examination question different composition representation and learner embedding | |
| CN112464004A (en) | Multi-view depth generation image clustering method | |
| CN113361685B (en) | Knowledge tracking method and system based on learner knowledge state evolution expression | |
| CN113190654A (en) | Knowledge graph complementing method based on entity joint embedding and probability model | |
| CN114491039B (en) | Primitive learning few-sample text classification method based on gradient improvement | |
| CN112116137A (en) | Student class dropping prediction method based on mixed deep neural network | |
| CN116383401A (en) | Knowledge graph completion method integrating text description and graph convolution mechanism | |
| CN117201122A (en) | Unsupervised attribute network anomaly detection method and system based on view level graph comparison learning | |
| CN115544158A (en) | Multi-knowledge-point dynamic knowledge tracking method applied to intelligent education system | |
| CN115170874A (en) | Self-distillation implementation method based on decoupling distillation loss | |
| CN113283488B (en) | Learning behavior-based cognitive diagnosis method and system | |
| CN111985560B (en) | Knowledge tracking model optimization method, system and computer storage medium | |
| CN113194493A (en) | Wireless network data missing attribute recovery method and device based on graph neural network | |
| CN112329918A (en) | Anti-regularization network embedding method based on attention mechanism | |
| CN111126860B (en) | Task allocation method, task allocation device and electronic equipment | |
| CN114971066A (en) | Knowledge tracking method and system integrating forgetting factor and learning ability | |
| CN114139674A (en) | Behavior cloning method, electronic device, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |