CN114780879A - Interpretable link prediction method for knowledge hypergraph - Google Patents

Interpretable link prediction method for knowledge hypergraph Download PDF

Info

Publication number
CN114780879A
CN114780879A CN202210324786.7A CN202210324786A CN114780879A CN 114780879 A CN114780879 A CN 114780879A CN 202210324786 A CN202210324786 A CN 202210324786A CN 114780879 A CN114780879 A CN 114780879A
Authority
CN
China
Prior art keywords
hypergraph
knowledge
tuple
hidden
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210324786.7A
Other languages
Chinese (zh)
Inventor
王鑫
陈子睿
王晨旭
刘鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN202210324786.7A priority Critical patent/CN114780879A/en
Publication of CN114780879A publication Critical patent/CN114780879A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • G06F16/9558—Details of hyperlinks; Management of linked annotations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Algebra (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses an interpretable link prediction method for a hypergraph, which comprises the steps of constructing an interpretable hypergraph representation learning model based on a hypergraph embedding model and a Markov logic network; establishing joint probability for all observable tuples and hidden tuples of the knowledge hypergraph through a Markov logic network, and taking the logarithm likelihood of the maximized observable tuples as a training target; optimizing a confidence lower bound of a log-likelihood function by adopting a variational EM algorithm to realize the training and verification of the model; and performing link prediction on the hypergraph data set by using the verified interpretable hypergraph representation learning model, namely taking a hidden tuple in the hypergraph data set as the input of the model, and outputting a probability value of the establishment of the hidden tuple and the contribution degree of an entity and a relation connected with the hidden tuple to the establishment of the hidden tuple by the model. By utilizing the method, the domain knowledge in the logic rule and the semantic information in the vector space can be fully utilized, and the effect of knowledge hypergraph representation learning is improved.

Description

Interpretable link prediction method for knowledge hypergraph
Technical Field
The invention relates to a knowledge hypergraph, in particular to large-scale knowledge hypergraph-oriented representation learning.
Background
With the rapid development of the internet, the data volume is explosively increasing. To deeply understand semantic information behind a user query and further enhance search quality of a search engine, Google corporation first proposed a concept of Knowledge Graph (Knowledge Graph) in 2012. Knowledge maps formally describe things in the real world and their relationships to each other, and are large-scale semantic networks that store human knowledge in the form of graphs. It represents knowledge as a triple p (s, o), where p is the predicate, s is the subject, and o is the object. A triple p (s, o) is used to indicate that there is a relationship p between the resource s and the resource o, or the resource s has an attribute p and its value is o.
A Hypergraph of Knowledge (KnowLEGE Hypergraph) is a graph-structured Knowledge base that stores the facts around the world in the form of multiple tuples and can be regarded as a generalization of a Knowledge graph, which represents Knowledge as n-tuples p (e)1,…,en-1) Where p is a predicate, eiIs the ith entity that makes up the tuple. One n-tuple for representing a resource eiHaving a relationship p. Due to the large number of facts in the real world, it is not practical to store all facts in the knowledge hypergraph. The biggest challenge to the prior knowledge hypergraph is its serious incompleteness, i.e. links between partial entities are missing. In the case of Freebase, 61% of the relationships involved are multivariate relationships, and more than one third of the entities stored are involved in the composition of multivariate relationships. In the face of the high imperfection of the knowledge hypergraph, manually adding links between entities is very labor-and material-consuming, and therefore, the requirement for an algorithm for automatically reasoning missing links between entities is generated.
Knowledge hypergraph representation learning aims at embedding entities and relationships as vectors of continuous low dimensions for efficient storage and computation. By using the vector representations, semantic association of entities and relations can be effectively represented, and the problems of low calculation efficiency and sparse data can be effectively solved. These characteristics of knowledge representation learning play an important role in the construction, reasoning and application of knowledge hypergraphs. m-TransH is a representative knowledge hypergraph representation learning method that projects entities and relationships into the same vector space. The m-TransH model is simple and efficient, and enjoyable results are obtained on the link prediction problem. Several enhanced m-TransH models, including RAE and NaLP, have been proposed by researchers to improve the ability of the knowledge hypergraph to predict and reason about. However, the knowledge hypergraph embedding method still presents two key challenges.
(1) The results of the link prediction are not interpretable. Most of the existing knowledge graph embedding methods are pure data-driven black box models, the contribution degree of a certain entity or relationship to a reasoning result cannot be clearly shown, and any information about the prediction reliability cannot be given. For areas where decisions such as banking, medical, legal, etc. can be of significant impact, it is important to give specific reasons for reasoning about the results. Therefore, how to become an urgent technical problem.
(2) The embedding method cannot be combined with domain knowledge of logic rules to accomplish reasoning. Most current presentation learning research focuses on retaining semantic information of entities and relationships to efficiently predict missing n-tuples, however, one limitation is that they do not exploit logical rules that can compactly encode domain knowledge, which is useful in many applications including interpreting inference results. Therefore, synchronizing the embedding method with the logic rules to accomplish reasoning to simultaneously utilize semantic information in the embedding space and domain knowledge in the logic rules is an important direction for knowledge representation learning future research.
In summary, there is an urgent need for a new knowledge hypergraph representation learning method with interpretability and capable of simultaneously utilizing semantic information in an embedding space and domain knowledge in a logic rule, which solves two key challenges that the link prediction result has no interpretability and that the embedding method reasoning cannot be integrated into the domain knowledge in the prior art.
Disclosure of Invention
Aiming at the prior art, the knowledge hypergraph has interpretability along with the development of the knowledge hypergraph, the knowledge hypergraph integrated with the field knowledge represents that the learning is an academic frontier problem of artificial intelligence, and the knowledge hypergraph has very high academic value and potential application value. The invention aims to provide an interpretable link prediction method for a hypergraph knowledge, wherein a model simultaneously uses a hypergraph knowledge embedding method and a Markov logic network to carry out hypergraph knowledge representation learning and uses logic rules to interpret an inference result, so that the interpretability of the inference result is realized and the representation performance of the hypergraph knowledge is improved.
In order to solve the above technical problem, the present invention provides an interpretable link prediction method for a knowledge hypergraph, comprising the following steps:
step one, constructing an interpretable knowledge hypergraph representation learning model based on a knowledge hypergraph embedding model and a Markov logic network;
establishing joint probability for all observable tuples and hidden tuples of the knowledge hypergraph through a Markov logic network, and taking the logarithm likelihood of the maximum observable tuple as a training target; optimizing the confidence lower bound of the log likelihood function by adopting a variational EM algorithm, wherein the method comprises the following steps: firstly, carrying out variation E step reasoning on the probability of the establishment of the hidden tuple, optimizing the parameters of the knowledge hypergraph embedded model, then carrying out M steps, adjusting the logic rule weight of the Markov logic network according to the hidden tuple establishment probability value obtained by variation E step reasoning, and circularly iterating the variation E step and the M steps according to the sequence to finish the training and verification of the interpretable knowledge hypergraph representation learning model;
step three, link prediction is carried out on the knowledge hypergraph data set by using the interpretable knowledge hypergraph representation learning model trained and verified in the step two, namely, a hidden tuple in the knowledge hypergraph data set is used as the input of the model, and the output of the model is as follows: the probability value of the establishment of the hidden tuple and the contribution degree of the entity and the relation connected with the hidden tuple to the establishment of the hidden tuple.
Further, the invention provides an interpretable link prediction method for a knowledge hypergraph, wherein:
the content of the first step is as follows: and inputting observable tuples of the hypergraph knowledge into the hypergraph knowledge embedding model and the Markov logic network respectively, and simultaneously inputting logic rule data corresponding to the observable tuples of the hypergraph knowledge into the Markov logic network.
In the second step, the variation EM algorithm consists of a variation E step and a variation M step, and the variation E step and the variation M step are iterated circularly according to the sequence to finish the training and verification of the interpretable knowledge hypergraph representation learning model; when the variation E step is executed, the knowledge in the logic rule is merged into the knowledge hypergraph embedding model, and the parameters of the knowledge hypergraph embedding model are optimized; and when the M steps are executed, combining the semantic information embedded in the space with the logic rule, and adjusting the logic rule weight of the Markov logic network.
In the invention, the content for optimizing the parameters of the knowledge hypergraph embedded model comprises the following steps of;
2-1) adjusting a variation distribution optimization function and embedding the knowledge hypergraph into a model to be integrated into variation E step training;
2-2) obtaining the real posterior distribution of the hidden tuple by using a Markov logic network, and optimizing the calculation process of the Markov blanket by adopting a sampling mode;
2-3) optimizing parameter values of the knowledge hypergraph embedding model by minimizing KL divergence of the variational distribution and the true posterior distribution.
In the present invention, the content of adjusting the logic rule weight of the markov logic network includes:
3-1) adopting a pseudo-likelihood function as an optimization object, and maximizing a log-likelihood function by optimizing the pseudo-likelihood function to adjust a logic rule weight value;
3-2) calculating the gradient of the logic rule by adopting a random gradient descent method, and updating the weight value.
Compared with the prior art, the invention has the beneficial effects that:
(1) the invention designs an interpretable link prediction method for a knowledge hypergraph aiming at a large-scale knowledge hypergraph, which is a knowledge representation learning model constructed on the basis of a knowledge hypergraph embedding model and a Markov logic network at the same time, adopts a variational EM algorithm to circularly iterate a variational E step and an M step to respectively complete parameter optimization of the knowledge hypergraph embedding model and logic rule weight adjustment of the Markov logic network, and after iterative training, the two steps converge and tend to be consistent aiming at joint distribution defined by all tuples, and the logic rule of the Markov logic network can be used for explaining the contribution degree of each entity and relation of the knowledge hypergraph to establishment of an inference tuple, thereby effectively improving the link prediction performance while realizing interpretability of an inference result.
(2) Compared with a pure data driven knowledge hypergraph representation learning model, the invention integrates logic rules based on the Markov logic network, so that the model can obtain the entities and relations related to the reasoning result in the Markov network through the Markov blanket and uses the logic rule weights related to the entity relations to explain the reasoning result. On the basis of the supergraph knowledge embedding method, the invention explores a method for integrating the logic rules into the supergraph knowledge embedding model, fully combines the advantages of the two models, can simultaneously utilize the semantic information of the embedding space and the domain knowledge of the logic rules to carry out the reasoning work of the hidden tuples, and can better complete the supergraph knowledge link prediction.
Drawings
FIG. 1 is a diagram of an example of a knowledge hypergraph, associated logic rules and Markov carpet in accordance with the present invention;
FIG. 2 is a flow chart of the interpretable knowledge hypergraph representation learning of the present invention;
FIG. 3 is an experimental result of link prediction on a knowledge hypergraph data set by the present invention and related methods of the prior art;
FIGS. 4-1 and 4-2 are experimental results of link prediction on a knowledge-graph dataset according to the present invention and related methods of the prior art;
FIGS. 5-1 and 5-2 are experimental results of link prediction performed on a knowledge hypergraph dataset using only the variational E step and (according to the invention) using the EM step, respectively, in combination with the knowledge hypergraph embedding method;
FIGS. 6-1 through 6-4 show the results of experiments performed on a knowledge-graph dataset to perform link prediction using only the variational E step and (in the present invention) the EM step, respectively, in conjunction with the Hypermap embedding method.
Detailed Description
The invention will be further described with reference to the following figures and specific examples, which are not intended to limit the invention in any way.
The invention provides an interpretable link prediction method for a knowledge hypergraph, which mainly comprises the following steps:
step one, constructing an interpretable knowledge hypergraph representation learning model based on a knowledge hypergraph embedding model and a Markov logic network, and comprising the following steps of: and inputting observable tuples of the hypergraph to the hypergraph embedding model, and simultaneously inputting observable tuples of the hypergraph and logic rule data corresponding to the observable tuples to the Markov logic network.
Step two, training and verifying the interpretable knowledge hypergraph representation learning model constructed in the step one, wherein the training and verifying step comprises the following steps: and establishing joint probabilities for all observable tuples and hidden tuples of the knowledge hypergraph through a Markov logic network, and taking the logarithm likelihood of the maximum observable tuple as a training target. In order to combine the knowledge hypergraph embedding model with the Markov logic network using the variational EM algorithm and simultaneously to iteratively train, the lower confidence bound of the log-likelihood function is optimized instead.
And optimizing the confidence lower bound of the log likelihood function by adopting a variational EM algorithm, wherein the variational EM algorithm consists of a variational E step and an M step. Firstly, executing variation E step reasoning hidden tuple establishment probability to optimize the parameters of the knowledge hypergraph embedded model, then executing M steps, adjusting the logic rule weight of the Markov logic network according to the hidden tuple establishment probability value obtained by variation E step reasoning, and circularly iterating the variation E step and the M steps according to the sequence to finish the training and verification of the interpretable knowledge hypergraph representation learning model.
When the variation E step is executed, the knowledge in the logic rule is merged into the knowledge hypergraph embedding model, and the parameters of the knowledge hypergraph embedding model are optimized, wherein the method comprises the following steps:
1) adjusting a variation distribution optimization function and combining with a spread reasoning to enable the hypergraph knowledge embedding model to carry out reasoning on a single hidden tuple, so that the hypergraph knowledge embedding model is integrated into a variation E-step training;
2) obtaining the real posterior distribution of the hidden tuples by using a Markov logic network, and optimizing the calculation process of the Markov blanket by adopting tuple indicator variable values related to the hidden tuples;
3) and calculating the real posterior distribution of the tuple by using the current parameter values of the knowledge hypergraph embedded model, and optimizing the parameter values of the knowledge hypergraph embedded model by minimizing KL divergence of the variational distribution and the real posterior distribution.
When the M steps are executed, semantic information in the embedding space is combined with the logic rules, and the logic rule weight of the Markov logic network is adjusted according to the tuple establishment probability value, wherein the method comprises the following steps:
1) adopting a pseudo-likelihood function as an optimization object to avoid calculating a distribution function related to a large number of hidden tuple indicating variable values, and maximizing a log-likelihood function by optimizing the pseudo-likelihood function to adjust a logic rule weight value;
2) and calculating the gradient of the logic rule by adopting a random gradient descent method, and updating the weight value.
Step three, using the interpretable hypergraph representation learning model trained and verified in the step two to carry out link prediction on a hypergraph data set, namely taking a hidden tuple in the hypergraph data set as the input of the model, finding all tuples connected with the hidden tuple and related entities and relations through the Markov blanket of the hidden tuple, calculating the contribution degree of each hidden tuple through the weight of a logic rule, and outputting the model: the probability value of the establishment of the hidden tuple and the contribution degree of the entity and the relation connected with the hidden tuple to the establishment of the hidden tuple.
By utilizing the interpretable link prediction method for the knowledge hypergraph, provided by the invention, the domain knowledge in the logic rule and the semantic information in the vector space can be fully utilized, and the effect of representing and learning the knowledge hypergraph is improved.
Example 1
The interpretable hypergraph of knowledge constructed in the invention represents a learning model, and as shown in fig. 1, an example diagram of a hypergraph of knowledge, associated logic rules and markov carpet is given. The circle represents an entity, the upper graph ellipse represents a plurality of n-tuple of the knowledge hypergraph, the lower graph ellipse represents a group, the n-tuple of the knowledge hypergraph consists of a plurality of entities, wherein parentsOf represents a parent-child relationship, couple represents a couple relationship, wasBornin represents a place-of-birth relationship, workAsFor represents a work identity relationship and consists of 3 entities, 2 entities, 3 entities and 3 entities respectively, rules based on predicates are shown in a middle graph Markov logic network, and the Markov logic network consists of logic rules and rule weights. In the variation E step, the knowledge hypergraph embedding method completes the prediction of a hidden tuple (such as serveAsFor (Lebron James, Basketball Player, marker) which does not exist in the knowledge hypergraph) by an embedded vector obtained by a left n-tuple, and in the M step, the rule weight of the Markov logic network is adjusted by taking an observable tuple and the hidden tuple predicted in the variation E step as training data. When the inference result is explained, the markov blanket of the inference result is used, taking a hidden tuple serveAsFor (Lebron James, basetball Player, Laker) as an example, the markov blanket displays all entities connected with the hidden tuple, the entities in a logic rule are connected with each other to form a group, all entities related to a closed rule are corresponding, and the contribution degree of each related entity and relationship to the establishment of the inference result can be explained through the weight of the logic rule obtained by M-step training.
Example 2
The interpretable connection prediction method provided by the invention is utilized to realize the process of knowledge hypergraph representation learning, as shown in FIG. 2, the process of the knowledge hypergraph representation learning method is as follows:
firstly, establishing an interpretable learning model for representing a knowledge hypergraph
1-1) input tuples and rule data, establish joint probabilities using Markov and maximize the log-likelihood of observable tuples.
Representing the knowledge hypergraph as
Figure BDA0003572943770000061
Wherein
Figure BDA0003572943770000062
ε、
Figure BDA0003572943770000063
And
Figure BDA0003572943770000064
respectively, a finite set of knowledge hypergraphs, entities, relationships, and observable tuples. Observable tuples ti＝r(e1，e2，...，ek) Wherein r is a relationship, each
Figure BDA0003572943770000065
For an entity, k is the non-negative integer number of the relation r and i is the observable tuple index. Each tuple tiAnd an indicator variable
Figure BDA0003572943770000066
The association is carried out in such a way that,
Figure BDA0003572943770000067
the representation tuple is true and,
Figure BDA0003572943770000068
then the tuple is denoted as false.
Given a set of logical rules L, the joint distribution of all tuples can be defined by the following equation (1):
Figure BDA0003572943770000069
wherein p is the joint distribution, Z is the partition function,
Figure BDA00035729437700000610
is the weight value of the ith logical rule,
Figure BDA00035729437700000611
is the ith set of closed rules
Figure BDA00035729437700000612
Size of (b)HIs a hidden tuple. The question whether the predictive hidden variable is true or not is given by the above formula (1)Question becomes a reasoning posterior distribution pt(bH|bO) The training of the Markov logic network by maximizing the observable tuples bOLog likelihood log p oft(bO) And (4) finishing.
1-2) optimizing the confidence lower bound of the log-likelihood function and introducing a variation EM algorithm.
In order to combine the knowledge hypergraph embedding model and the Markov logic network at the same time, the lower confidence bound of the log-likelihood in the step 1-1) is optimized, as shown in the formula (2):
Figure BDA00035729437700000613
wherein KL calculates KL divergence of true posterior and variational posterior, and P is calculated after variational posteriorvAnd true posterior ptWhen equal, the lower confidence bound can be effectively optimized by the variational EM algorithm.
The variation EM algorithm consists of a variation E step and an M step, when the variation E step is executed, the variation distribution is updated to reduce p while the real posterior distribution is fixedv(bH) And pt(bO，bH) The KL divergence of the logic rules can be integrated into the knowledge hypergraph embedding model in the process; when executing M steps, the variation distribution is fixed and the real posterior distribution is updated to maximize the log-likelihood function under all tuples
Figure BDA00035729437700000614
This process may combine semantic information embedded in space with logical rules.
Second, executing the probability of forming hidden tuple by variation E-step reasoning
2-1) obtaining the variation distribution of the hidden tuples by using a knowledge hypergraph embedding model.
The goal of the variation E step is to infer the hidden tuple bHTrue posterior distribution pt(bH|bO) But since precise reasoning is not feasible due to the complex graph structure of the hypergraph knowledge, by using the mean field variation distributionTo approximate the true posterior distribution such that the variation distribution pv(bH) Reasoning can be done for each hidden tuple t independently.
The method for predicting hidden tuples by embedding a knowledge hypergraph model is to learn observable tuples bOBased on these embeddings, the joint distribution of all tuples can be defined as the following equation (3):
Figure BDA00035729437700000615
wherein Bern denotes bernoulli distribution, and the scoring function f calculates a tuple t ═ r (e) based on the relation embedding r and the entity embedding e1，e2，...，ek) Is the probability of being correct. Maximizing the log probability log p (b) by stochastic gradient descent with observable tuples as positive examples and implicit tuples as negative examplesO＝1，bH0) to efficiently optimize the model parameters.
In order to combine the Markov logic network with the knowledge hypergraph embedding model, the variation distribution p is distributed by using the spreading reasoningv(bH) The parameterization is the parameter of the knowledge hypergraph embedding model, so that the joint distribution of formula (3) can be combined with the Markov logic network in the variation E step, and the reasoning formula (4) is as follows:
Figure BDA0003572943770000071
2-2) obtaining true posterior distribution of hidden tuples using Markov logic network
By minimizing pv(bH) And pt(bH|bO) The KL divergence between the two, the probability value of the Markov logic network predicting the hidden tuple is carried out by a fixed formula (5):
Figure BDA0003572943770000072
where mb (t) is a markov carpet for tuple t, const is a fixed constant, and any regular closed rule set tuple, as long as it occurs concurrently with tuple t, can be found in the markov carpet mb (t).
The expectation in equation (5) may be by sampling
Figure BDA0003572943770000073
To simplify the computation, such that for each tuple t 'in the Markov carpet, if t' is observable
Figure BDA0003572943770000074
Otherwise
Figure BDA0003572943770000075
Taking the value of variation distribution calculated by embedding the knowledge hypergraph into the model, so that the variation distribution can be further simplified into
Figure BDA0003572943770000076
By the step 2-1) and the execution of this step, the value b of the indicator variable of each hidden tuple ttThe prediction can be carried out by a hypergraph knowledge embedded model and a Markov logic network at the same time, if any one t' connected with the tuple t is a hidden tuple, an indicating variable is available, and the simplification of the distribution of the variation is combined
Figure BDA0003572943770000077
It can be known that the prediction result of the knowledge hypergraph embedding model should remain the same as that of the markov logic network.
2-3) optimizing parameter values of the knowledge hypergraph embedding model.
The parameter mu of the learning knowledge hypergraph model can be obtained by an objective function O of the following formula (6)μCarrying out the following steps:
Figure BDA0003572943770000078
the objective function can be calculated from the current μ value
Figure BDA0003572943770000079
And updating μ with the true posterior distribution as a target to minimize KL divergence of the variation distribution and the true distribution. Such domain knowledge encoded in the logic rules can be incorporated into the knowledge hypergraph embedding model.
Thirdly, adjusting the weight of the logic rule according to the tuple establishment probability value by executing the M steps
And 3-1) updating the weight value by optimizing the pseudo likelihood function.
In step M, to solve the partition function in formula (1), we turn to the adjustment of the logic rule weights by fixing the parameters of the knowledge hypergraph embedded model and maximizing the log-likelihood to optimize the pseudo-likelihood function, which is shown in formula (7):
Figure BDA0003572943770000081
3-2) calculating the gradient of the logical rule weight using a random gradient descent.
For the expectation of equation (7) in step 3-1), for the rule l that the hidden tuple is connected in the markov carpet, the gradient of its weight can be calculated by the random gradient descent of equation (8):
Figure BDA0003572943770000082
Figure BDA0003572943770000083
is a rule weight wlIf the tuple t is observable, the probability y of being satisfied for the tuple t in equation (8)t1, otherwise, taking the variation posterior pv(bt＝1)，
Figure BDA0003572943770000084
Is a sampling from a variational posteriori, for each tuple t' in it, if it is observable
Figure BDA0003572943770000085
Otherwise get
Figure BDA0003572943770000086
Finally, for each observable tuple, an attempt is made to maximize
Figure BDA0003572943770000087
For each hidden tuple, pv(bt1) as a target to update the probability value
Figure BDA0003572943770000088
Thus, the semantic information learned by the knowledge hypergraph embedding model can be combined with the logic rules.
Fourthly, outputting the probability of establishing the hidden tuple and the contribution degree of each entity relation to establishing the hidden tuple
For each prediction tuple t, in an arbitrary logical rule l, the tuple t' that occurs simultaneously with t is r (e)1，e2，...，ek) May be found in markov blanket mb (t). All entities and relationships that appear in the Markov blanket can be grouped into sets ε' and ε
Figure BDA0003572943770000089
Tuple set
Figure BDA00035729437700000810
The system consists of all tuples in the Markov carpet containing an entity e ', and the contribution degree of each entity e' to the tuple t can be calculated by the formula (9):
Figure BDA00035729437700000811
wherein the content of the first and second substances,
Figure BDA00035729437700000812
tuple set obtainable in the same manner as described above
Figure BDA00035729437700000813
The degree of contribution of each relation r' is calculated. With the contribution of each entity and relationship, the respective percentage of contribution can be obtained by dividing the sum by the contribution of the type, and for the contribution of an entity to the predicted result under a specific one of the relationships r' e l or the contribution of a relationship to the predicted result under a specific one of the rules l, the respective percentage of contribution can be obtained by the rule weight values of the trained markov logic network.
Example 3
The method of the invention and the related method in the prior art carry out the link prediction experiment result on the hypergraph knowledge data set or the knowledge map data set.
The invention carries out link prediction on a knowledge hypergraph data set JF17K, M-FB15K and FB-AUTO and knowledge map data sets FB15k, WN18, FB15k-237 and WN18 RR.
Referring to FIG. 3, on the knowledge hypergraph dataset, m-CP is selected as the embedding model of the knowledge hypergraph. Compared with pure m-CP and other knowledge hypergraph embedding methods, the technical scheme of the invention obtains better performance on almost all evaluation indexes. The reason is that the knowledge hypergraph embedding method only utilizes semantic information in an embedding space, and the invention predicts the establishment probability value of the hidden tuple discovered by the Markov logic network by further combining the embedding of the domain knowledge in the logic rule and further updates the weight of the logic rule to achieve better performance. From the results of the FB-AUTO data set, the invention achieves the same effect compared with the pure m-CP. This is mainly related to the size of the FB-AUTO dataset (7 relations, 11213 tuples total), with no logical rules contributing when searching for hidden tuples. Therefore, the intersection between the test set and the set of hidden tuples is empty, which results in that the included logic rules cannot be refined into the knowledge hypergraph embedding, resulting in the same composite effect as the pure embedding method.
Referring to FIGS. 4-1 and 4-2, on the knowledge-graph dataset, m-DistMult was chosen as the embedding model for the hypergraph. Similar to the result on the hypergraph knowledge, the invention is applied to the knowledge graph, and a better result is obtained in the common knowledge graph with the binary relation, thereby proving the effectiveness of the invention. Because the pure Markov logic network is only applied to the binary relation, the invention only compares the reasoning performance with the pure Markov logic network on the knowledge graph data set. Similar to the knowledge hypergraph dataset, the experimental results show the efficient performance of the model created in the present invention on these evaluation indices.
Fig. 5-1, 5-2, and 6-1 through 6-4 illustrate the results of training the present invention with the variational EM algorithm and its variants (only with the variational E step), respectively. In variation E step, the embedding model learns the domain knowledge from the logic rules, while in M step, the weights of the logic rules may be optimized by the learned embedding. When the present invention combines the same embedding methods in the same dataset, the effect of using both the variation E and M steps is generally higher than the effect of using only the variation E step.
Although the present invention has been described in connection with the accompanying drawings, the present invention is not limited to the above-described embodiments, which are intended to be illustrative rather than restrictive, and many modifications may be made by those skilled in the art without departing from the spirit of the present invention as disclosed in the appended claims.

Claims (5)

1. An interpretable link prediction method for a knowledge hypergraph, comprising the steps of:
step one, constructing an interpretable knowledge hypergraph representation learning model based on a knowledge hypergraph embedding model and a Markov logic network;
establishing joint probability for all observable tuples and hidden tuples of the knowledge hypergraph through a Markov logic network, and taking the logarithm likelihood of the maximum observable tuple as a training target; optimizing the confidence lower bound of the log likelihood function by adopting a variational EM algorithm, wherein the method comprises the following steps: firstly, carrying out variation E step reasoning on the probability of the establishment of the hidden tuple, optimizing the parameters of the knowledge hypergraph embedded model, then carrying out M steps, adjusting the logic rule weight of the Markov logic network according to the hidden tuple establishment probability value obtained by variation E step reasoning, and circularly iterating the variation E step and the M steps according to the sequence to finish the training and verification of the interpretable knowledge hypergraph representation learning model;
step three, using the interpretable hypergraph representation learning model trained and verified in the step two to perform link prediction on the hypergraph data set, namely taking a hidden tuple in the hypergraph data set as the input of the model, wherein the output of the model is as follows: the probability value of the establishment of the hidden tuple and the contribution degree of the entity and the relation connected with the hidden tuple to the establishment of the hidden tuple.
2. The interpretable link prediction method for a knowledge hypergraph of claim 1, wherein the content of step one is: and inputting observable tuples of the hypergraph knowledge into the hypergraph knowledge embedding model and the Markov logic network respectively, and simultaneously inputting logic rule data corresponding to the observable tuples of the hypergraph knowledge into the Markov logic network.
3. The interpretable link prediction method for the knowledge hypergraph according to claim 1, wherein in step two, the variation EM algorithm is composed of a variation E step and a variation M step, and the variation E step and the variation M step are iterated according to the sequence loop to complete training and verification of the interpretable knowledge hypergraph representation learning model;
when the variation E step is executed, the knowledge in the logic rule is merged into the knowledge hypergraph embedding model, and the parameters of the knowledge hypergraph embedding model are optimized;
and when the M steps are executed, combining the semantic information embedded in the space with the logic rules, and adjusting the logic rule weight of the Markov logic network.
4. The interpretable link prediction method for a knowledge hypergraph according to claim 1 or 2, wherein the content for optimizing the parameters of the knowledge hypergraph embedding model includes;
2-1) adjusting a variation distribution optimization function and integrating a knowledge hypergraph embedded model into variation E-step training;
2-2) obtaining the real posterior distribution of the hidden tuples by using a Markov logic network, and optimizing the calculation process of the Markov blanket by adopting a sampling mode;
2-3) optimizing parameter values of the knowledge hypergraph embedding model by minimizing KL divergence of the variational and true posterior distributions.
5. The interpretable link prediction method for a knowledgegraph according to claim 1 or 2, wherein the adjusting of the logic rule weights of the markov logic network comprises:
3-1) adopting a pseudo-likelihood function as an optimization object, and maximizing a log-likelihood function by optimizing the pseudo-likelihood function to adjust a logic rule weight value;
3-2) calculating the gradient of the logic rule by adopting a random gradient descent method, and updating the weight value.
CN202210324786.7A 2022-03-30 2022-03-30 Interpretable link prediction method for knowledge hypergraph Pending CN114780879A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210324786.7A CN114780879A (en) 2022-03-30 2022-03-30 Interpretable link prediction method for knowledge hypergraph

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210324786.7A CN114780879A (en) 2022-03-30 2022-03-30 Interpretable link prediction method for knowledge hypergraph

Publications (1)

Publication Number Publication Date
CN114780879A true CN114780879A (en) 2022-07-22

Family

ID=82428083

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210324786.7A Pending CN114780879A (en) 2022-03-30 2022-03-30 Interpretable link prediction method for knowledge hypergraph

Country Status (1)

Country Link
CN (1) CN114780879A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115757806A (en) * 2022-09-21 2023-03-07 清华大学 Hyper-relation knowledge graph embedding method and device, electronic equipment and storage medium
CN117273143A (en) * 2023-08-08 2023-12-22 南京邮电大学 Time sequence knowledge graph reasoning method and system based on Markov logic network

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115757806A (en) * 2022-09-21 2023-03-07 清华大学 Hyper-relation knowledge graph embedding method and device, electronic equipment and storage medium
CN115757806B (en) * 2022-09-21 2024-05-28 清华大学 Super-relationship knowledge graph embedding method and device, electronic equipment and storage medium
CN117273143A (en) * 2023-08-08 2023-12-22 南京邮电大学 Time sequence knowledge graph reasoning method and system based on Markov logic network

Similar Documents

Publication Publication Date Title
CN112653142B (en) Wind power prediction method and system for optimizing depth transform network
CN109948029B (en) Neural network self-adaptive depth Hash image searching method
CN109783817B (en) Text semantic similarity calculation model based on deep reinforcement learning
CN112529168B (en) GCN-based attribute multilayer network representation learning method
Jin et al. Empowering graph representation learning with test-time graph transformation
CN114780879A (en) Interpretable link prediction method for knowledge hypergraph
CN106411896A (en) APDE-RBF neural network based network security situation prediction method
Chen et al. Rlpath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning
CN112905801A (en) Event map-based travel prediction method, system, device and storage medium
CN112988917A (en) Entity alignment method based on multiple entity contexts
Xu et al. Graph partitioning and graph neural network based hierarchical graph matching for graph similarity computation
Mu et al. Auto-CASH: A meta-learning embedding approach for autonomous classification algorithm selection
CN113190688A (en) Complex network link prediction method and system based on logical reasoning and graph convolution
Zhang et al. Learn to abstract via concept graph for weakly-supervised few-shot learning
Wei et al. Autoias: Automatic integrated architecture searcher for click-trough rate prediction
Szwarcman et al. Quantum-inspired evolutionary algorithm applied to neural architecture search
CN112580728A (en) Dynamic link prediction model robustness enhancing method based on reinforcement learning
Czajkowski et al. Steering the interpretability of decision trees using lasso regression-an evolutionary perspective
Lei et al. DigGCN: Learning compact graph convolutional networks via diffusion aggregation
Zhang et al. Reinforcement learning with actor-critic for knowledge graph reasoning
Lyu et al. A survey of model compression strategies for object detection
Rasekh et al. EDNC: Evolving differentiable neural computers
Hruschka et al. Towards efficient imputation by nearest-neighbors: A clustering-based approach
CN116975434A (en) Content recommendation method and related equipment
CN116208399A (en) Network malicious behavior detection method and device based on metagraph

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination