CN111597358A - Knowledge graph reasoning method and device based on relational attention and computer equipment - Google Patents

Knowledge graph reasoning method and device based on relational attention and computer equipment Download PDF

Info

Publication number
CN111597358A
CN111597358A CN202010708066.1A CN202010708066A CN111597358A CN 111597358 A CN111597358 A CN 111597358A CN 202010708066 A CN202010708066 A CN 202010708066A CN 111597358 A CN111597358 A CN 111597358A
Authority
CN
China
Prior art keywords
neighbor
embedded representation
representation
embedded
target node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010708066.1A
Other languages
Chinese (zh)
Inventor
赵晓娟
陈恺
常春喜
王培�
王昌海
马锶霞
李晨晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202010708066.1A priority Critical patent/CN111597358A/en
Publication of CN111597358A publication Critical patent/CN111597358A/en
Priority to CN202011284767.3A priority patent/CN112528035B/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
    • G06F16/367—Ontology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a knowledge graph reasoning method and device based on relational attention and computer equipment. The method comprises the following steps: obtaining initial embedded representation of nodes in a knowledge graph, converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation, obtaining a neighbor node set of target nodes in the knowledge graph, constructing neighbor subgraphs according to relationship types, obtaining neighbor embedded representation of information in the target nodes embedded in the neighbor subgraphs according to the high-dimensional embedded representation of the target nodes and the high-dimensional embedded representation of the neighbor nodes in the neighbor subgraphs, aggregating the high-dimensional embedded representation of the target nodes and the neighbor embedded representation to obtain aggregated embedded representation of the target nodes, fusing the aggregated embedded representations according to a first attention score of the neighbor subgraphs to obtain fused embedded representation of the target nodes, calculating scores of corresponding to the target nodes according to the fused embedded representation, and performing triple reasoning according to the scores. The method can improve the precision of the triple reasoning.

Description

Knowledge graph reasoning method and device based on relational attention and computer equipment
Technical Field
The present application relates to the field of knowledge graph technology, and in particular, to a knowledge graph inference method, apparatus, computer device, and storage medium based on relationship attention.
Background
Knowledge maps play a very important role in many natural language processing applications, such as question-answering systems, semantic searches, and the like. However, due to uncertainty of knowledge acquisition, the knowledge graph constructed based on the entity identification and relationship extraction technology may cause incompleteness of the knowledge graph, thereby affecting the performance of these applications. If there is an error in the knowledge-graph, this may cause the application to return an erroneous result. The knowledge graph reasoning technology refers to deducing new facts according to known facts in the existing knowledge graph, and the knowledge graph can be enriched by using the knowledge graph reasoning technology.
A triplet is a basic unit of knowledge representation in a knowledge graph and is used to represent a relationship between an entity and an entity, or what is an attribute value of some attribute of an entity. For example: we use nodes in the knowledge graph
Figure 987321DEST_PATH_IMAGE001
Representing, for nodes, relationships between nodes
Figure 879053DEST_PATH_IMAGE002
The knowledge graph can be represented as
Figure 414640DEST_PATH_IMAGE003
Wherein each triplet describes an entity and some relationship between entities.
In order to capture the complex and hidden information inherent in the local neighborhood around the triples, in the prior relevant research, a model based on graph attention succeeds in obtaining a network structure, but is not suitable for being directly used in a knowledge graph because the model ignores a part of important information in the knowledge graph, namely relationship information between entities in KG. One end-to-end model that considers relationships in Knowledge maps, proposed in document 1 (Nathani D, Chauhan J, Sharma C, et al, Learning Attention-based rules for relationship Prediction in Knowledge maps [ C ]//. recording of the association for computational relationships, (ACL) 2019: 4710-4723.), takes into account a relationship vector representation at each level of node feature aggregation. But the model only considers simple splicing of the relationship and the entity representation in the calculation of attention and does not highlight the importance of the relationship. The GAT model proposed in document 2 (Velikovi P, Cucurull G, Casanova A, et al, Graph Attention networks [ C ]// 6th International Conference on Learning retrieval, { ICLR } 2018, Vancouver, BC, Canada, April 30-May 3, 2018.) considers all neighbors, giving each neighbor a different weight, but GAT only considers first-order neighbors and does not concern the edge features. All result in inaccurate knowledge graph reasoning.
Disclosure of Invention
In view of the above, it is necessary to provide a relation attention-based knowledge graph reasoning method, apparatus, computer device and storage medium capable of reasoning inaccuracy problem for the knowledge graph.
A relationship attention-based knowledge-graph inference method, the method comprising:
acquiring initial embedded representation of nodes in a knowledge graph, and converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation;
acquiring a neighbor node set of a target node in the knowledge graph, and constructing a neighbor sub-graph according to the relationship type of the target node and the neighbor node in the neighbor node set;
obtaining neighbor embedded representation of the information embedded into the neighbor subgraph by the target node according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
according to the first attention value of each neighbor subgraph, fusing the aggregation embedded representation to obtain a fused embedded representation of the target node;
and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
In one embodiment, the method further comprises the following steps: embedding a head entity, a tail entity and a relation of a triple in a knowledge graph according to a preset embedding model to obtain an initial embedding representation; and converting the head entity and the tail entity into a high-dimensional space according to a preset conversion matrix to obtain a head entity high-dimensional embedded representation, and converting the relationship into the high-dimensional space according to a preset relationship conversion matrix to obtain a relationship high-dimensional embedded representation.
In one embodiment, the method further comprises the following steps: obtaining the high-dimensional embedded representation of the target node and the node pair embedded representation of the neighbor node in the neighbor subgraph as follows:
Figure 518862DEST_PATH_IMAGE004
wherein the content of the first and second substances,
Figure 940617DEST_PATH_IMAGE005
the pair of representation nodes is represented in an embedded manner,
Figure 144196DEST_PATH_IMAGE006
a first linear transformation matrix is represented and,
Figure 284190DEST_PATH_IMAGE007
a high-dimensional embedded representation representing the target node,
Figure 379054DEST_PATH_IMAGE008
a node pair embedded representation representing a neighbor node in the neighbor subgraph;
determining, according to the node pair embedded representation, a second attention score of the neighbor node to the target node as:
Figure 920894DEST_PATH_IMAGE009
wherein the content of the first and second substances,
Figure 623271DEST_PATH_IMAGE010
a second attention score is indicated which is,
Figure 125927DEST_PATH_IMAGE011
it is shown that the activation function is,
Figure 634269DEST_PATH_IMAGE012
representing a second linear transformation matrix;
and normalizing the second attention value, and aggregating to obtain a neighbor embedding representation according to the normalization result of the second attention value of each neighbor node in the neighbor subgraph.
In one embodiment, the method further comprises the following steps: according to a preset aggregation function, aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregate embedded representation of the target node, wherein the aggregate embedded representation of the target node is as follows:
Figure 233878DEST_PATH_IMAGE013
wherein the content of the first and second substances,
Figure 529992DEST_PATH_IMAGE014
the representation is aggregated into an embedded representation,
Figure 910158DEST_PATH_IMAGE015
the representation neighbor embeds the representation.
In one embodiment, the method further comprises the following steps: calculating a first attention score for each of the neighboring subgraphs as:
Figure 894294DEST_PATH_IMAGE016
wherein the content of the first and second substances,
Figure 286092DEST_PATH_IMAGE017
the corresponding relation type of the neighbor subgraph is represented,
Figure 392589DEST_PATH_IMAGE018
representing a diagonal of
Figure 119105DEST_PATH_IMAGE019
The diagonal matrix of (a) is,
Figure 906933DEST_PATH_IMAGE020
to represent
Figure 543450DEST_PATH_IMAGE021
An embedded representation of (a);
and normalizing the first attention value, and fusing to obtain a fusion embedding expression according to the normalization result of the first attention value of the neighbor subgraph.
In one embodiment, the method further comprises the following steps: expanding the fused embedded representation to obtain a multi-head attention embedded representation as follows:
Figure 758531DEST_PATH_IMAGE022
wherein M represents the number of heads of attention;
and according to the multi-head attention embedding expression, expanding to obtain a multi-layer embedding expression as follows:
Figure 457497DEST_PATH_IMAGE023
where k represents the number of layers.
In one embodiment, the method further comprises the following steps: calculating the score of the triple corresponding to the target node according to the fusion embedded representation; if the scores of the triples are within the corresponding evaluation index preset ranking range, determining that the triples are established; and if the scores of the triples are not within the preset ranking range of the corresponding evaluation indexes, determining that the triples are false.
A relationship attention-based knowledge-graph inference apparatus, the apparatus comprising:
the high-dimensional embedding module is used for acquiring initial embedding representation of nodes in the knowledge graph, and converting the initial embedding representation into a high-dimensional space to obtain high-dimensional embedding representation;
the subgraph construction module is used for acquiring a neighbor node set of a target node in the knowledge graph and constructing a neighbor subgraph according to the relationship type of the target node and the neighbor node in the neighbor node set;
the neighbor information embedding module is used for obtaining neighbor embedded representation of the information of the target node embedded in the neighbor subgraph according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
the aggregation module is used for aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
the fusion module reasoning module is used for fusing the aggregation embedded representation according to the first attention value of each neighbor subgraph to obtain a fusion embedded representation of the target node; and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring initial embedded representation of nodes in a knowledge graph, and converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation;
acquiring a neighbor node set of a target node in the knowledge graph, and constructing a neighbor sub-graph according to the relationship type of the target node and the neighbor node in the neighbor node set;
obtaining neighbor embedded representation of the information embedded into the neighbor subgraph by the target node according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
according to the first attention value of each neighbor subgraph, fusing the aggregation embedded representation to obtain a fused embedded representation of the target node;
and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring initial embedded representation of nodes in a knowledge graph, and converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation;
acquiring a neighbor node set of a target node in the knowledge graph, and constructing a neighbor sub-graph according to the relationship type of the target node and the neighbor node in the neighbor node set;
obtaining neighbor embedded representation of the information embedded into the neighbor subgraph by the target node according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
according to the first attention value of each neighbor subgraph, fusing the aggregation embedded representation to obtain a fused embedded representation of the target node;
and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
According to the knowledge graph reasoning method, the device, the computer equipment and the storage medium based on the relation attention, the nodes are embedded into the high-dimensional space, then the neighbor subgraphs are determined according to the neighbor nodes of the target nodes when the target nodes are embedded, then the neighbor embedded representation of the target nodes embedded into the neighbor subgraph information is determined according to the relation between the target nodes and the connecting edges of the nodes in the neighbor subgraphs, on the other hand, the high-dimensional embedded representation and the neighbor embedded representation are aggregated, then the attention scores are adopted for all the neighbor subgraphs to be fused, and the fused embedded representation is obtained, so that the reasoning of the knowledge graph is performed. Because the information of the connecting edges of the fusion neighbor nodes is fused and embedded, the accuracy of knowledge graph reasoning can be improved.
Drawings
FIG. 1 is a flow diagram of a method for relation attention based knowledge-graph inference in one embodiment;
FIG. 2 is a block diagram of a knowledge-graph inference engine based on relational attention in one embodiment;
FIG. 3 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, as shown in FIG. 1, a method for relation attention based knowledge-graph inference is provided, comprising the steps of:
and 102, acquiring initial embedded representation of nodes in the knowledge graph, and converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation.
Representing the nodes in the knowledge graph by adopting the triples, and representing the triples
Figure 845753DEST_PATH_IMAGE024
Wherein, in the step (A),
Figure 959946DEST_PATH_IMAGE025
the header entity is represented as a header entity,
Figure 345928DEST_PATH_IMAGE026
the relationship is represented by a relationship of,
Figure 922403DEST_PATH_IMAGE027
representing the tail entity, and the initial embedding is obtained by inputting the entities and the relations in the triples into a preset word embedding model. The conversion of the initial embedded representation into a high dimensional space may be achieved by a high dimensional conversion matrix.
And 104, acquiring a neighbor node set of the target node in the knowledge graph, and constructing a neighbor sub-graph according to the relationship type of the target node and the neighbor node in the neighbor node set.
In the knowledge-graph, there are nodes connected to the target node, and all the nodes connected to the target node constitute neighbor nodes of the target node.
In this embodiment, the neighboring nodes are clustered according to the types of the connecting edges of the target node and the neighboring nodes, that is, the neighboring nodes of the same type construct a neighboring subgraph.
And 106, obtaining the neighbor embedded representation of the information embedded into the neighbor subgraph by the target node according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph.
And step 108, aggregating the high-dimensional embedded representation and the neighbor embedded representation of the target node to obtain an aggregated embedded representation of the target node.
And step 110, fusing the aggregation embedded representations according to the first attention value of each neighbor subgraph to obtain a fused embedded representation of the target node.
In this embodiment, an attention mechanism is introduced, that is, for each neighbor sub-graph, a first attention score is calculated, so as to set a fusion weight for each neighbor sub-graph.
And 112, calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
According to the knowledge graph reasoning method based on the relational attention, the nodes are embedded into a high-dimensional space, then when the target node is embedded, the neighbor subgraphs are determined according to the neighbor nodes of the target node, then the neighbor embedded representation of the information of the target node embedded neighbor subgraphs is determined according to the relation between the target node and the connecting edges of the nodes in the neighbor subgraphs, on the other hand, the high-dimensional embedded representation and the neighbor embedded representation are aggregated, and then the attention values are adopted for fusion aiming at all the neighbor subgraphs to obtain the fusion embedded representation, so that the reasoning of the knowledge graph is carried out. Because the information of the connecting edges of the fusion neighbor nodes is fused and embedded, the accuracy of knowledge graph reasoning can be improved.
In one embodiment, embedding a head entity, a tail entity and a relation of a triple in a knowledge graph according to a preset embedding model to obtain an initial embedding representation; and according to a preset relation conversion matrix, converting the relation into a high-dimensional space to obtain a relation high-dimensional embedded representation.
In particular, in triplets
Figure 989716DEST_PATH_IMAGE028
For example, the initial embedded representation may be:
Figure 69667DEST_PATH_IMAGE029
wherein the content of the first and second substances,
Figure 892130DEST_PATH_IMAGE030
。
converting the initial embedded representation to a high-dimensional embedded representation as:
Figure 814955DEST_PATH_IMAGE031
wherein the content of the first and second substances,
Figure 810593DEST_PATH_IMAGE032
,
Figure 620417DEST_PATH_IMAGE033
are respectively triplets
Figure 613781DEST_PATH_IMAGE034
The initial embedded vector.
Figure 164848DEST_PATH_IMAGE035
,
Figure 59117DEST_PATH_IMAGE036
Is node embedding after mapping to a high-dimensional space,
Figure 316923DEST_PATH_IMAGE037
is a relational embedding that maps to a high dimensional space.
Figure 543505DEST_PATH_IMAGE038
Is a matrix of the transitions,
Figure 253972DEST_PATH_IMAGE039
is a relational transformation matrix.
In another embodiment, obtaining the high-dimensional embedded representation of the target node and the node pair embedded representation of the neighbor nodes in the neighbor subgraph is as follows:
Figure 201200DEST_PATH_IMAGE040
wherein the content of the first and second substances,
Figure 641408DEST_PATH_IMAGE041
the pair of representation nodes is represented in an embedded manner,
Figure 632367DEST_PATH_IMAGE042
a first linear transformation matrix is represented and,
Figure 892447DEST_PATH_IMAGE043
a high-dimensional embedded representation representing the target node,
Figure 705682DEST_PATH_IMAGE044
node pairs representing neighbor nodes in a neighbor subgraph embed a representation.
Determining a second attention score of the neighbor node to the target node according to the node pair embedded representation as follows:
Figure 610184DEST_PATH_IMAGE045
wherein the content of the first and second substances,
Figure 912989DEST_PATH_IMAGE046
a second attention score is indicated which is,
Figure 289394DEST_PATH_IMAGE047
it is shown that the activation function is,
Figure 703058DEST_PATH_IMAGE048
representing a second linear transformation matrix; and normalizing the second attention value, and aggregating to obtain a neighbor embedding representation according to the normalization result of the second attention value of each neighbor node in the neighbor subgraph.
Specifically, the normalization process is as follows:
Figure 789962DEST_PATH_IMAGE049
wherein, the normalization adopts a softmax function,
Figure 139035DEST_PATH_IMAGE050
representing a set of neighbor nodes.
The aggregation results in a neighbor embedding representation as:
Figure 108128DEST_PATH_IMAGE051
wherein the content of the first and second substances,
Figure 450117DEST_PATH_IMAGE052
is shown in the relation
Figure 657107DEST_PATH_IMAGE053
The lower neighbors are embedded in the representation.
In one embodiment, according to a preset aggregation function, aggregating the high-dimensional embedded representation and the neighbor embedded representation of the target node to obtain an aggregate embedded representation of the target node, where:
Figure 36136DEST_PATH_IMAGE054
wherein the content of the first and second substances,
Figure 633471DEST_PATH_IMAGE055
the representation is aggregated into an embedded representation,
Figure 388937DEST_PATH_IMAGE056
the representation neighbor embeds the representation.
In another embodiment, the first attention score for each neighboring subgraph is calculated as:
Figure 450434DEST_PATH_IMAGE057
wherein the content of the first and second substances,
Figure 157621DEST_PATH_IMAGE058
the corresponding relation type of the neighbor subgraph is represented,
Figure 101306DEST_PATH_IMAGE059
representing a diagonal of
Figure 598147DEST_PATH_IMAGE060
The diagonal matrix of (a) is,
Figure 451833DEST_PATH_IMAGE060
to represent
Figure 438244DEST_PATH_IMAGE061
An embedded representation of (a); normalizing the first attention score and according to the neighbor subgraphThe result of the normalization of the first attention score is fused to obtain a fused embedded representation. It is worth mentioning that knowledge graph reasoning involves multiple relationships
Figure 72487DEST_PATH_IMAGE062
It is understood as an embedded representation of a plurality of combinations of relationships.
The process of normalizing the first attention score is:
Figure 294390DEST_PATH_IMAGE063
the fusion results in a fusion embedding represented as:
Figure 127217DEST_PATH_IMAGE064
in one embodiment, the fused embedded representation is expanded to obtain a multi-head attention embedded representation as:
Figure 487791DEST_PATH_IMAGE065
wherein M represents the number of heads of attention;
and according to the multi-head attention embedding representation, expanding to obtain a multi-layer embedding representation as follows:
Figure 281435DEST_PATH_IMAGE066
where k represents the number of layers.
In one embodiment, the score of the target node corresponding to the triplet is calculated according to the fused embedded representation; if the scores of the triples are within the corresponding evaluation index preset ranking range, determining that the triples are established; and if the scores of the triples are not within the preset ranking range of the corresponding evaluation indexes, determining that the triples are false.
Specifically, the evaluation index can adopt an MMR index, and the ranking can be realized through a Hit @ N algorithm.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 2, there is provided a relation attention-based knowledge-graph inference apparatus, comprising: a high-dimensional embedding module 202, a subgraph construction module 204, a neighbor information embedding module 206, an aggregation module 208, and a fusion module inference module 210, wherein:
a high-dimensional embedding module 202, configured to obtain an initial embedded representation of a node in a knowledge graph, and convert the initial embedded representation into a high-dimensional space to obtain a high-dimensional embedded representation;
a subgraph construction module 204, configured to obtain a neighbor node set of a target node in the knowledge graph, and construct a neighbor subgraph according to a relationship type between the target node and a neighbor node in the neighbor node set;
a neighbor information embedding module 206, configured to obtain a neighbor embedded representation of information in the target node embedded in the neighbor subgraph according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
an aggregation module 208, configured to aggregate the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregate embedded representation of the target node;
the fusion module reasoning module 210 is configured to fuse the aggregate embedded representation according to the first attention score of each neighboring sub-graph to obtain a fusion embedded representation of the target node; and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
In one embodiment, the high-dimensional embedding module 202 is further configured to embed a head entity, a tail entity, and a relationship of a triple in the knowledge graph according to a preset embedding model, so as to obtain an initial embedding representation; and according to a preset relation conversion matrix, converting the relation into a high-dimensional space to obtain a relation high-dimensional embedded representation.
In one embodiment, the neighbor information embedding module 206 is further configured to obtain a high-dimensional embedded representation of the target node and a node pair embedded representation of a neighbor node in the neighbor subgraph as:
Figure 916816DEST_PATH_IMAGE067
wherein the content of the first and second substances,
Figure 492897DEST_PATH_IMAGE068
the pair of representation nodes is represented in an embedded manner,
Figure 821110DEST_PATH_IMAGE069
a first linear transformation matrix is represented and,
Figure 836471DEST_PATH_IMAGE070
a high-dimensional embedded representation representing the target node,
Figure 541121DEST_PATH_IMAGE071
a node pair embedded representation representing a neighbor node in the neighbor subgraph;
determining, according to the node pair embedded representation, a second attention score of the neighbor node to the target node as:
Figure 286224DEST_PATH_IMAGE072
wherein the content of the first and second substances,
Figure 175551DEST_PATH_IMAGE073
a second attention score is indicated which is,
Figure 802841DEST_PATH_IMAGE074
it is shown that the activation function is,
Figure 655391DEST_PATH_IMAGE075
representing a second linear transformation matrix;
and normalizing the second attention value, and aggregating to obtain a neighbor embedding representation according to the normalization result of the second attention value of each neighbor node in the neighbor subgraph.
In one embodiment, the aggregating module 208 is further configured to aggregate the high-dimensional embedded representation of the target node and the neighbor embedded representation according to a preset aggregation function, and obtain an aggregate embedded representation of the target node as:
Figure 520579DEST_PATH_IMAGE076
wherein the content of the first and second substances,
Figure 456174DEST_PATH_IMAGE077
the representation is aggregated into an embedded representation,
Figure 196859DEST_PATH_IMAGE078
the representation neighbor embeds the representation.
In one embodiment, the fusion module inference module 210 is further configured to calculate the first attention score of each of the neighboring subgraphs as:
Figure 180995DEST_PATH_IMAGE079
wherein the content of the first and second substances,
Figure 963007DEST_PATH_IMAGE080
representing correspondences of neighbour subgraphsThe type of the relationship is,
Figure 7186DEST_PATH_IMAGE081
representing a diagonal of
Figure 484435DEST_PATH_IMAGE082
The diagonal matrix of (a) is,
Figure 334579DEST_PATH_IMAGE082
to represent
Figure 830151DEST_PATH_IMAGE083
An embedded representation of (a);
and normalizing the first attention value, and fusing to obtain a fusion embedding expression according to the normalization result of the first attention value of the neighbor subgraph.
In one embodiment, the fusion module inference module 210 is further configured to expand the fusion embedded representation to obtain a multi-head attention embedded representation as:
Figure 45232DEST_PATH_IMAGE084
wherein M represents the number of heads of attention;
and according to the multi-head attention embedding expression, expanding to obtain a multi-layer embedding expression as follows:
Figure 868832DEST_PATH_IMAGE085
where k represents the number of layers.
In one embodiment, the fusion module inference module 210 is further configured to calculate a score of the triple corresponding to the target node according to the fusion embedded representation; if the scores of the triples are within the corresponding evaluation index preset ranking range, determining that the triples are established; and if the scores of the triples are not within the preset ranking range of the corresponding evaluation indexes, determining that the triples are false.
The specific definition of the relation attention-based knowledge graph inference device can refer to the above definition of the relation attention-based knowledge graph inference method, and is not described in detail herein. The modules in the knowledge-graph inference apparatus based on relationship attention can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 3. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of relation attention based knowledge-graph inference. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method in the above embodiments when the processor executes the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method in the above-mentioned embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A relationship attention-based knowledge-graph inference method, the method comprising:
acquiring initial embedded representation of nodes in a knowledge graph, and converting the initial embedded representation into a high-dimensional space to obtain high-dimensional embedded representation;
acquiring a neighbor node set of a target node in the knowledge graph, and constructing a neighbor sub-graph according to the relationship type of the target node and the neighbor node in the neighbor node set;
obtaining neighbor embedded representation of the information embedded into the neighbor subgraph by the target node according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
according to the first attention value of each neighbor subgraph, fusing the aggregation embedded representation to obtain a fused embedded representation of the target node;
and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
2. The method of claim 1, wherein obtaining an initial embedded representation of a node in the knowledge-graph, converting the initial embedded representation to a high-dimensional space, resulting in a high-dimensional embedded representation, comprises:
embedding a head entity, a tail entity and a relation of a triple in a knowledge graph according to a preset embedding model to obtain an initial embedding representation;
and converting the head entity and the tail entity into a high-dimensional space according to a preset conversion matrix to obtain a head entity high-dimensional embedded representation, and converting the relationship into the high-dimensional space according to a preset relationship conversion matrix to obtain a relationship high-dimensional embedded representation.
3. The method of claim 1, wherein deriving a neighbor-embedded representation of information in the target node-embedded neighbor sub-graph from the high-dimensional embedded representation of the target node and the high-dimensional embedded representations of neighbor nodes in the neighbor sub-graph comprises:
obtaining the high-dimensional embedded representation of the target node and the node pair embedded representation of the neighbor node in the neighbor subgraph as follows:
Figure 464108DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 390082DEST_PATH_IMAGE002
the pair of representation nodes is represented in an embedded manner,
Figure 871879DEST_PATH_IMAGE003
a first linear transformation matrix is represented and,
Figure 426488DEST_PATH_IMAGE004
a high-dimensional embedded representation representing the target node,
Figure 310131DEST_PATH_IMAGE005
a node pair embedded representation representing a neighbor node in the neighbor subgraph;
determining, according to the node pair embedded representation, a second attention score of the neighbor node to the target node as:
Figure 908471DEST_PATH_IMAGE006
wherein the content of the first and second substances,
Figure 80827DEST_PATH_IMAGE007
a second attention score is indicated which is,
Figure 298181DEST_PATH_IMAGE008
it is shown that the activation function is,
Figure 505172DEST_PATH_IMAGE009
representing a second linear transformation matrix;
and normalizing the second attention value, and aggregating to obtain a neighbor embedding representation according to the normalization result of the second attention value of each neighbor node in the neighbor subgraph.
4. The method of claim 1, wherein aggregating the high-dimensional embedded representation of the target node with the neighbor embedded representation to obtain an aggregated embedded representation of the target node comprises:
according to a preset aggregation function, aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregate embedded representation of the target node, wherein the aggregate embedded representation of the target node is as follows:
Figure 25146DEST_PATH_IMAGE010
wherein the content of the first and second substances,
Figure 747114DEST_PATH_IMAGE011
the representation is aggregated into an embedded representation,
Figure 705843DEST_PATH_IMAGE012
the representation neighbor embeds the representation.
5. The method of claim 4, wherein fusing the aggregated embedded representation according to the first attention score of each of the neighboring subgraphs to obtain a fused embedded representation of the target node, comprises:
calculating a first attention score for each of the neighboring subgraphs as:
Figure 190176DEST_PATH_IMAGE013
wherein the content of the first and second substances,
Figure 5685DEST_PATH_IMAGE014
the corresponding relation type of the neighbor subgraph is represented,
Figure 152633DEST_PATH_IMAGE015
representing a diagonal of
Figure 852736DEST_PATH_IMAGE016
The diagonal matrix of (a) is,
Figure 565477DEST_PATH_IMAGE017
to represent
Figure 489570DEST_PATH_IMAGE018
An embedded representation of (a);
and normalizing the first attention value, and fusing to obtain a fusion embedding expression according to the normalization result of the first attention value of the neighbor subgraph.
6. The method according to any one of claims 1 to 5, further comprising:
expanding the fused embedded representation to obtain a multi-head attention embedded representation as follows:
Figure 310765DEST_PATH_IMAGE019
wherein M represents the number of heads of attention;
and according to the multi-head attention embedding expression, expanding to obtain a multi-layer embedding expression as follows:
Figure 408034DEST_PATH_IMAGE020
where k represents the number of layers.
7. The method according to any of claims 1 to 5, wherein calculating scores of triples corresponding to the target node based on the fused embedded representation, and performing triplet inference based on the scores comprises:
calculating the score of the triple corresponding to the target node according to the fusion embedded representation;
if the scores of the triples are within the corresponding evaluation index preset ranking range, determining that the triples are established;
and if the scores of the triples are not within the preset ranking range of the corresponding evaluation indexes, determining that the triples are false.
8. A relationship attention-based knowledge-graph inference apparatus, the apparatus comprising:
the high-dimensional embedding module is used for acquiring initial embedding representation of nodes in the knowledge graph, and converting the initial embedding representation into a high-dimensional space to obtain high-dimensional embedding representation;
the subgraph construction module is used for acquiring a neighbor node set of a target node in the knowledge graph and constructing a neighbor subgraph according to the relationship type of the target node and the neighbor node in the neighbor node set;
the neighbor information embedding module is used for obtaining neighbor embedded representation of the information of the target node embedded in the neighbor subgraph according to the high-dimensional embedded representation of the target node and the high-dimensional embedded representation of the neighbor node in the neighbor subgraph;
the aggregation module is used for aggregating the high-dimensional embedded representation of the target node and the neighbor embedded representation to obtain an aggregated embedded representation of the target node;
the fusion module reasoning module is used for fusing the aggregation embedded representation according to the first attention value of each neighbor subgraph to obtain a fusion embedded representation of the target node; and calculating the score of the triple corresponding to the target node according to the fusion embedding representation, and performing triple reasoning according to the score.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202010708066.1A 2020-07-22 2020-07-22 Knowledge graph reasoning method and device based on relational attention and computer equipment Pending CN111597358A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010708066.1A CN111597358A (en) 2020-07-22 2020-07-22 Knowledge graph reasoning method and device based on relational attention and computer equipment
CN202011284767.3A CN112528035B (en) 2020-07-22 2020-11-17 Knowledge graph reasoning method and device based on relational attention and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010708066.1A CN111597358A (en) 2020-07-22 2020-07-22 Knowledge graph reasoning method and device based on relational attention and computer equipment

Publications (1)

Publication Number Publication Date
CN111597358A true CN111597358A (en) 2020-08-28

Family

ID=72188250

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202010708066.1A Pending CN111597358A (en) 2020-07-22 2020-07-22 Knowledge graph reasoning method and device based on relational attention and computer equipment
CN202011284767.3A Active CN112528035B (en) 2020-07-22 2020-11-17 Knowledge graph reasoning method and device based on relational attention and computer equipment

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202011284767.3A Active CN112528035B (en) 2020-07-22 2020-11-17 Knowledge graph reasoning method and device based on relational attention and computer equipment

Country Status (1)

Country Link
CN (2) CN111597358A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112348190A (en) * 2020-10-26 2021-02-09 福州大学 Uncertain knowledge graph prediction method based on improved embedded model SUKE
CN112559737A (en) * 2020-11-20 2021-03-26 和美（深圳）信息技术股份有限公司 Node classification method and system of knowledge graph
CN113111134A (en) * 2021-04-21 2021-07-13 山东省人工智能研究院 Self-coding and attention mechanism-based heterogeneous graph node feature embedding method
CN113254580A (en) * 2021-05-24 2021-08-13 厦门大学 Special group searching method and system
CN113780002A (en) * 2021-08-13 2021-12-10 北京信息科技大学 Knowledge reasoning method and device based on graph representation learning and deep reinforcement learning
CN116150392A (en) * 2022-12-12 2023-05-23 首都师范大学 Threat information knowledge graph processing method, threat information knowledge graph processing device, threat information knowledge graph processing equipment and storage medium

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112948506A (en) * 2021-04-01 2021-06-11 重庆邮电大学 Improved meta-learning relation prediction method based on convolutional neural network
CN114328956B (en) * 2021-12-23 2023-02-28 北京百度网讯科技有限公司 Text information determination method and device, electronic equipment and storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10877979B2 (en) * 2018-01-16 2020-12-29 Accenture Global Solutions Limited Determining explanations for predicted links in knowledge graphs
US10824949B2 (en) * 2018-09-27 2020-11-03 Babylon Partners Limited Method and system for extracting information from graphs
CN110825884B (en) * 2019-11-05 2024-05-24 腾讯科技（深圳）有限公司 Embedded representation processing method and device based on artificial intelligence and electronic equipment
CN110866190B (en) * 2019-11-18 2021-05-14 支付宝(杭州)信息技术有限公司 Method and device for training neural network model for representing knowledge graph
CN110929047B (en) * 2019-12-11 2023-04-28 中国人民解放军国防科技大学 Knowledge graph reasoning method and device for focusing on neighbor entity

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112348190A (en) * 2020-10-26 2021-02-09 福州大学 Uncertain knowledge graph prediction method based on improved embedded model SUKE
CN112348190B (en) * 2020-10-26 2022-06-21 福州大学 Uncertain knowledge graph prediction method based on improved embedded model SUKE
CN112559737A (en) * 2020-11-20 2021-03-26 和美（深圳）信息技术股份有限公司 Node classification method and system of knowledge graph
CN113111134A (en) * 2021-04-21 2021-07-13 山东省人工智能研究院 Self-coding and attention mechanism-based heterogeneous graph node feature embedding method
CN113254580A (en) * 2021-05-24 2021-08-13 厦门大学 Special group searching method and system
CN113254580B (en) * 2021-05-24 2023-10-03 厦门大学 Special group searching method and system
CN113780002A (en) * 2021-08-13 2021-12-10 北京信息科技大学 Knowledge reasoning method and device based on graph representation learning and deep reinforcement learning
CN113780002B (en) * 2021-08-13 2023-11-28 北京信息科技大学 Knowledge reasoning method and device based on graph representation learning and deep reinforcement learning
CN116150392A (en) * 2022-12-12 2023-05-23 首都师范大学 Threat information knowledge graph processing method, threat information knowledge graph processing device, threat information knowledge graph processing equipment and storage medium

Also Published As

Publication number Publication date
CN112528035A (en) 2021-03-19
CN112528035B (en) 2022-09-16

Similar Documents

Publication Publication Date Title
CN112528035B (en) Knowledge graph reasoning method and device based on relational attention and computer equipment
CN110929047A (en) Knowledge graph reasoning method and device concerning neighbor entities
Chang et al. Factorized similarity learning in networks
CN111651671B (en) User object recommendation method, device, computer equipment and storage medium
Wu et al. A novel community answer matching approach based on phrase fusion heterogeneous information network
CN112148892A (en) Method and device for complementing knowledge of dynamic knowledge graph and computer equipment
CN113268612A (en) Heterogeneous information network knowledge graph completion method and device based on mean value fusion
Xu et al. Robust network enhancement from flawed networks
CN116804997A (en) Chinese similar case recommending method, device and equipment based on graph neural network
CN113566831B (en) Unmanned aerial vehicle cluster navigation method, device and equipment based on human-computer interaction
CN116304341A (en) Fraud discrimination method and system based on user network big data
CN111428009A (en) Relationship query method and device, computer equipment and storage medium
CN115062779A (en) Event prediction method and device based on dynamic knowledge graph
CN111813949B (en) Network space knowledge graph reasoning method and device for combined query
Xing et al. Mining semantic information in rumor detection via a deep visual perception based recurrent neural networks
CN109582802B (en) Entity embedding method, device, medium and equipment
Tong Research on multiple classification detection for network traffic anomaly based on deep learning
Adewole et al. The quadratic entropy approach to implement the Id3 decision tree algorithm
Li et al. Causal Reinforcement Learning for Knowledge Graph Reasoning
CN111476037B (en) Text processing method and device, computer equipment and storage medium
CN114357977B (en) Method, system, equipment and storage medium for realizing anti-plagiarism
CN117909517A (en) Knowledge graph completion method, apparatus, device, storage medium, and program product
Rehman et al. A scalable model for real‐time venue recommendations using MapReduce
Bing et al. Hcrpc: Highly compact reachability preserving graph compression with corrections
CN116467466A (en) Knowledge graph-based code recommendation method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200828

WD01 Invention patent application deemed withdrawn after publication