CN113626610A - Knowledge graph embedding method and device, computer equipment and storage medium - Google Patents
Knowledge graph embedding method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113626610A CN113626610A CN202110912715.4A CN202110912715A CN113626610A CN 113626610 A CN113626610 A CN 113626610A CN 202110912715 A CN202110912715 A CN 202110912715A CN 113626610 A CN113626610 A CN 113626610A
- Authority
- CN
- China
- Prior art keywords
- entity
- sample
- semantic
- features
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000000605 extraction Methods 0.000 claims abstract description 96
- 238000012545 processing Methods 0.000 claims abstract description 19
- 239000013598 vector Substances 0.000 claims description 84
- 238000013507 mapping Methods 0.000 claims description 55
- 239000007787 solid Substances 0.000 claims description 22
- 238000012549 training Methods 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 13
- 239000000126 substance Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000013604 expression vector Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The application relates to the technical field of artificial intelligence, and provides a knowledge graph embedding method, a knowledge graph embedding device, computer equipment and a storage medium. The knowledge graph embedding effect can be improved. The method comprises the following steps: the method comprises the steps of obtaining entity structural features and relation features of a knowledge graph by using a pre-constructed basic feature extraction model, wherein the entity structural features at least comprise head entity structural features and tail entity structural features, obtaining an entity description text of the knowledge graph, processing the entity description text by using the pre-constructed semantic feature extraction model to obtain entity semantic features, the entity semantic features at least comprise head entity semantic features and tail entity semantic features, and embedding the knowledge graph by using the head entity structural features, the tail entity structural features, the relation features, the entity semantic features and the tail entity semantic features.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for embedding a knowledge graph, a computer device, and a storage medium.
Background
The knowledge map stores a piece of knowledge describing a fact in the real world. Knowledge in a knowledge-graph is typically represented in the form of triples, such as (head entities, relationships, tail entities) and (entities, attributes, values). An entity is generally a name of a thing, a name of a concept, etc. in the real world, and a relationship is a description of an association between an entity and an entity. The knowledge-graph represents entities by nodes and relationships between entities by edges between the entities.
Knowledge graph embedding is an important basis of other applications based on knowledge graphs, and the purpose of the knowledge graph embedding is to mainly perform vector representation on entities, relations and other information forming the knowledge graphs and simultaneously reserve original characteristics of the entities, the relations and the like in the knowledge graphs as much as possible.
The current technology mainly carries out entity and relationship embedding by restricting a head entity vector and a relationship vector to be equal to a tail entity vector, but because the technology processes the knowledge graph simply, only the structural characteristics of a triple are basically learned, and the technical problem of poor knowledge graph embedding effect exists.
Disclosure of Invention
In view of the above, it is necessary to provide a knowledge graph embedding method, apparatus, computer device and storage medium for solving the above technical problems.
A method of knowledge-graph embedding, the method comprising:
acquiring entity structural features and relationship features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features;
acquiring an entity description text of the knowledge graph, and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
and embedding the knowledge map by using the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
A knowledge-graph embedding apparatus, comprising:
the first feature extraction module is used for acquiring the entity structure features and the relation features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features;
the second feature extraction module is used for acquiring an entity description text of the knowledge graph and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
and the embedding processing module is used for embedding the knowledge graph by utilizing the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring entity structural features and relationship features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features; acquiring an entity description text of the knowledge graph, and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features; and embedding the knowledge map by using the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring entity structural features and relationship features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features; acquiring an entity description text of the knowledge graph, and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features; and embedding the knowledge map by using the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
The method, the device, the computer equipment and the storage medium for embedding the knowledge map utilize a pre-constructed basic feature extraction model to obtain the entity structural features and the relationship features of the knowledge map, the entity structural features at least comprise head entity structural features and tail entity structural features, obtain the entity description text of the knowledge map and process the entity description text by utilizing the pre-constructed semantic feature extraction model to obtain entity semantic features, the entity semantic features at least comprise head entity semantic features and tail entity semantic features, and the knowledge map is embedded by utilizing the head entity structural features, the tail entity structural features, the relationship features, the entity semantic features and the tail entity semantic features. The method comprehensively considers the relation of the knowledge graph and the semantics of the entity, uses a basic feature extraction model to extract the structural features of the relation and the entity, and uses a semantic feature extraction model to extract the semantic features of the entity, thereby integrally improving the embedding effect of the knowledge graph.
Drawings
FIG. 1 is a schematic flow chart diagram of a method for knowledge graph embedding in one embodiment;
FIG. 2 is a block diagram of the knowledge-graph embedding apparatus in one embodiment;
FIG. 3 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The knowledge graph embedding method provided by the application can be executed by computer equipment such as a terminal or a server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server can be implemented by an independent server or a server cluster formed by a plurality of servers.
In one embodiment, as shown in FIG. 1, a knowledge-graph embedding method is provided, which may include the steps of:
s101, acquiring entity structure characteristics and relationship characteristics of a knowledge graph by using a pre-constructed basic characteristic extraction model;
the method mainly comprises the step of extracting the entity structure characteristics and the relation characteristics of the knowledge graph by using a pre-constructed basic characteristic extraction model, wherein the entity structure characteristics at least comprise head entity structure characteristics and tail entity structure characteristics, and in some embodiments, a transD model can be used as the basic characteristic extraction model to extract the entity structure characteristics and the relation characteristics of the knowledge graph. Specifically, the method comprises the steps of initializing the entity and the relation of the knowledge graph by using a pre-constructed basic feature extraction model, giving a triple (h, r, t) of the knowledge graph, and obtaining the structural features of a head entity, the structural features of a relation and a tail entity of the basic feature extraction model respectively by using the basic feature extraction model
For the knowledge graph containing N1 entities and N2 relations, the structural features, the relation features and the tail entity features of the head entity, the relation features and the tail entity, which are extracted by the basic feature extraction model, are respectively
And
wherein n and m represent the characteristic dimensions of the entity and the relationship, respectively.
Step S102, acquiring an entity description text of the knowledge graph, and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features, wherein the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
the method mainly comprises the step of obtaining entity semantic features by utilizing a pre-constructed semantic feature extraction model. Specifically, the knowledge graph not only represents the relationship between two entities in a triple form, but also comprises a text description of the entities, namely an entity description text.
In some embodiments, the processing the entity description text by using the pre-constructed semantic feature extraction model in step S102 to obtain the entity semantic features specifically includes:
segmenting words of the entity description text and expressing the word vectors to obtain a first word vector sequence corresponding to the entity description text; and inputting the first word vector sequence into a semantic feature extraction model so that the semantic feature extraction model obtains semantic relevancy between each entity vector contained in the first word vector sequence and other word vectors, obtains a second word vector sequence corresponding to the entity description text according to the semantic relevancy, each entity vector and other word vectors, and obtains entity semantic features according to the second word vector sequence.
Specifically, after an entity description text is segmented and word vector representation is performed, a word vector sequence (i.e., a first word vector sequence) corresponding to the entity description text is obtained as x1,x2,…,xKWherein the k-th position is an entity vector x corresponding to the entity description textk. Then the word vector sequence is input into a semantic feature extraction model, and the semantic feature extraction model acquires all the words contained in the first word vector sequenceSemantic relevance of the entity vector and other word vectors respectively, and then obtaining a second word vector sequence g corresponding to the entity description text according to the semantic relevance, each entity vector and other word vectors1,g2,…,gKObtaining entity semantic features including head entity semantic features according to the second word vector sequence
And tail entity semantic features
In some embodiments, the semantic feature extraction model may specifically include a self-attention model and a convolutional neural network, where the self-attention model may be specifically configured to obtain semantic correlations between each entity vector included in the first word vector sequence and other word vectors, and obtain a second word vector sequence corresponding to the entity description text according to the semantic correlations, each entity vector, and other word vectors; the convolutional neural network may be specifically configured to obtain the entity semantic features from the second word vector sequence obtained from the attention model.
The embodiment mainly combines a self-attention mechanism and a convolutional neural network to process an entity description text so as to enhance the extraction of entity semantic features. Specifically, a first word vector sequence x is set1,x2,…,xKInputting a self-attention model, wherein each entity vector x can be obtained by the self-attention modelkSemantic relatedness a with other word vectors respectivelyi：
score(xk,xi)＝vTtanh(W[xk:xi]),
Wherein, [ x ]k:xi]Representing the concatenation of the matrices, W and v are parameters of the neural networks of the first and second layers, respectively, and tanh represents the activation function. Followed by the attention processing of the self-attention modelThe subsequent expression vector of the entity description text is denoted as gkG ofkThe calculation process of (2) is as follows:
gk＝xk+∑i≠kaixi
the calculation process embodies the self-attention model according to the semantic relevance aiEach entity vector xkAnd other word vectors xiObtaining a second word vector sequence g corresponding to the entity description text1,g2,…,gK. Then, a second word vector sequence g learned for the self-attention model1,g2,…,gKInputting the convolutional neural network to obtain semantic features of the entity, i.e. semantic features of the head entity
And tail entity semantic features
And step S103, embedding the knowledge map by using the head entity structural features, the tail entity structural features, the relation features, the entity semantic features and the tail entity semantic features.
In the step, the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature can be used as the entity embedding feature and the relation feature of the knowledge graph to carry out knowledge graph reasoning.
The method for embedding the knowledge graph comprises the steps of acquiring entity structural features and relation features of the knowledge graph by using a pre-constructed basic feature extraction model, wherein the entity structural features at least comprise head entity structural features and tail entity structural features, acquiring an entity description text of the knowledge graph, processing the entity description text by using the pre-constructed semantic feature extraction model to obtain entity semantic features, wherein the entity semantic features at least comprise head entity semantic features and tail entity semantic features, and embedding the knowledge graph by using the head entity structural features, the tail entity structural features, the relation features, the entity semantic features and the tail entity semantic features. The method comprehensively considers the relation of the knowledge graph and the semantics of the entity, uses a basic feature extraction model to extract the structural features of the relation and the entity, and uses a semantic feature extraction model to extract the semantic features of the entity, thereby integrally improving the embedding effect of the knowledge graph.
The training process (or referred to as a building process) of the basic feature extraction model and the semantic feature extraction model is mainly described by the following embodiments, and in order to distinguish the terms from the terms in the model application stage, the relevant terms are expressed by using a "sample" in the model training stage. In some embodiments, the method provided herein may further include the steps of:
acquiring a knowledge graph sample, and inputting the knowledge graph sample into a basic feature extraction model to be trained so that the basic feature extraction model to be trained outputs a head entity structure feature sample, a head entity structure mapping sample, a tail entity structure feature sample, a tail entity structure mapping sample, a relation feature sample and a relation mapping sample; and training the basic feature extraction model to be trained by using a target function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample and the relation mapping sample.
In this embodiment, the basic feature extraction model to be trained may specifically be a transD model. Inputting the knowledge graph sample into a basic feature extraction model to be trained, wherein the basic feature extraction model to be trained can output: head solid structure characteristic sample
Head entity structure mapping samples
Tail entity structural feature sample
Tail entity structure mapping samples
Relational feature samples
And relational mapping samples
Wherein, N1 represents the number of entity samples, N2 represents the number of relationship samples, N and m represent the feature dimensions of the entity samples and the relationship samples, respectively, and the mapping between the entity samples and the relationship samples is used to calculate the mapping matrix.
Given a triple sample (h, r, t), its head entity structure feature sample, relation feature sample, tail entity structure feature sample and mapping sample can be expressed as
Next, a relationship matrix is defined
And
they can map entities from entity space to relationship space. Therefore, a projection vector h of the head entity and the tail entity can be obtained⊥,t⊥And a scoring function fr：
Wherein | h | purple2≤1,||t||2≤1,||r||2≤1,|h⊥|2≤1,|t⊥|2≤1。
Thus, the following objective function L can be minimized1Training the basic feature extraction model to be trained:
L1＝∑∑max(γ+fr(ξ)-fr(ξ′),0)
the training of the basic feature extraction model can adopt random gradient descent training to obtain a parameter gamma of the basic feature extraction model.
In an embodiment, the scheme provided by the present application may further include the following steps: acquiring an entity description text sample; inputting an entity description text sample into a semantic feature extraction model to be trained so that the semantic feature extraction model to be trained outputs a head entity semantic feature sample, a head entity semantic mapping sample, a tail entity semantic feature sample and a tail entity semantic mapping sample; in the above embodiment, the training of the basic feature extraction model to be trained by using the target function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relationship feature sample, and the relationship mapping sample specifically includes: and training the basic feature extraction model to be trained and the semantic feature extraction model to be trained by using an objective function constructed based on a head entity structure feature sample, a head entity structure mapping sample, a tail entity structure feature sample, a tail entity structure mapping sample, a relation feature sample, a relation mapping sample, a head entity semantic feature sample, a tail entity semantic feature sample and a tail entity semantic mapping sample.
In the embodiment, a new energy function is constructed to train the basic feature extraction model to be trained and the semantic feature extraction model to be trained in order to better integrate the structure and semantic features of the entity. Specifically, after the entity description text sample is obtained, the entity description text sample may be subjected to word segmentation and word vector representation to obtain a corresponding first word vector sequence sample x1,x2,…,xKThen, the word vector sequence sample can be specifically input into a self-attention model in the semantic feature extraction model, and a semantic relevancy sample a of the entity vector sample and other word vector samples is obtained from the self-attention modeli：
score(xk,xi)＝vTtanh(W[xk:xi]),
Then, the sample of the expression vector of the entity description text after the attention processing of the self-attention model is recorded as gkSimilarly, the gkThe calculation process of (2) is as follows: gk＝xk+∑i≠kaixi。
Then, the second word vector sequence sample g learned for the self-attention model1,g2,…,gKDesigning a convolutional neural network (four-layer network: word vector convolutional layer + pooling layer and nonlinear conversion layer + word vector convolutional layer + pooling layer and nonlinear conversion layer) in the semantic feature extraction model and orienting the second word vector sequence sample g1,g2,…,gKThe input convolutional neural network can obtain entity semantic feature samples output by the convolutional neural network, and the method comprises the following steps: head entity semantic feature samples
Head entity semantic mapping samples
Tail entity semantic feature sample
And tail entity semantic mapping samples
Wherein the network architecture of the four-layer network of the convolutional neural network may be the same, but their respective weights are initialized at random differently and the weights of the networks of the layers (Net)1、Net2、Net3And Net4) Will be updated during the model training process.
In order to better integrate the structure and semantic features of the entity, a head entity structure feature sample H and a head entity structure mapping sample H are obtainedpTail entity structure characteristic sample T and tail entity structure mapping sample TpHead entity semantic feature sample HdFirst entity semantic mapping sample HdpTail entity semantic feature sample TdTail entity semantic mapping sample TdpRelation characteristic sample R and relation mapping sample RpSetting a new energy function on the basis of the following steps:
wherein the content of the first and second substances,
is a relationship matrix that maps entities from an entity space to a relationship space. Thus, the scoring function f on the new triplet (h, r, t) can be derivedr-new：
Wherein the content of the first and second substances,
thereby, by minimizing the following objective function L2Training a basic feature extraction model to be trained and a semantic feature extraction model to be trained:
the training of the basic feature extraction model and the semantic feature extraction model can adopt random gradient descent training, so as to obtain the parameter gamma and the weight (Net) of each layer of network1、Net2、Net3And Net4)。
In order to further improve the training effect of the model, in some embodiments, the obtaining the knowledge-graph sample in the above embodiments specifically includes: acquiring a positive knowledge graph sample; carrying out random replacement processing on any element in the triple in the knowledge graph positive sample to construct a knowledge graph negative sample; and taking the knowledge graph positive sample and the knowledge graph negative sample as knowledge graph samples.
In this embodiment, the positive sample set Δ ═ h of the knowledge graph is obtainedi,ri,ti),(i＝1,2,…,nt) For training the model, and then constructing a negative sample set Δ', i.e., a non-existent triplet, on the basis thereof. During the construction of the negative sample set delta', the knowledge graph is constructed by the existing triples (h) in the positive samplej,rj,tj) Random substitution h in e deltaj,rj,tjObtaining nonexistent triples to construct a negative sample set delta' of the knowledge graph. And finally, taking the positive sample and the negative sample of the knowledge graph as the knowledge graph sample to train the model.
In the overall situation, the knowledge graph embedding method provided by the application can comprehensively consider the semantics of the relationship and the entity, can extract the structural features of the relationship and the entity by using a TransD model and extract the semantic features of the entity by using a self-attention mechanism and a convolutional neural network, so that the semantic information of the entity is simultaneously blended in during triple embedding to make up the defect of weak extraction capability of the semantic information features of the Transs series and reduce the complexity of the model as much as possible, and a new energy function is designed to integrate the semantic features and the structural features of the entity, thereby improving the overall knowledge graph embedding effect.
It should be understood that, although the steps in the above flowcharts are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above flowcharts may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the steps or the stages in other steps.
In one embodiment, as shown in FIG. 2, a knowledge-graph embedding apparatus is provided, the apparatus 200 may comprise:
the first feature extraction module 201 is configured to obtain an entity structure feature and a relationship feature of a knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features;
the second feature extraction module 202 is configured to obtain an entity description text of the knowledge graph, and process the entity description text by using a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
and the embedding processing module 203 is used for embedding the knowledge graph by using the head entity structural feature, the tail entity structural feature, the relationship feature, the entity semantic feature and the tail entity semantic feature.
In an embodiment, the second feature extraction module 202 is configured to perform word segmentation on the entity description text and perform word vector representation, so as to obtain a first word vector sequence corresponding to the entity description text; inputting the first word vector sequence into the semantic feature extraction model, so that the semantic feature extraction model obtains semantic relevancy between each entity vector contained in the first word vector sequence and other word vectors, obtains a second word vector sequence corresponding to the entity description text according to the semantic relevancy, each entity vector and other word vectors, and obtains the entity semantic features according to the second word vector sequence.
In one embodiment, the semantic feature extraction model includes a self-attention model and a convolutional neural network; the self-attention model is used for acquiring semantic relevance of each entity vector contained in the first word vector sequence and other word vectors, and obtaining a second word vector sequence corresponding to the entity description text according to the semantic relevance, each entity vector and other word vectors; and the convolutional neural network is used for acquiring the entity semantic features according to the second word vector sequence obtained by the self-attention model.
In one embodiment, the base feature extraction model is a transD model.
In one embodiment, the apparatus 200 may further include: the model training module is used for acquiring a knowledge graph sample; inputting the knowledge graph sample into a basic feature extraction model to be trained, so that the basic feature extraction model to be trained outputs a head entity structure feature sample, a head entity structure mapping sample, a tail entity structure feature sample, a tail entity structure mapping sample, a relation feature sample and a relation mapping sample; and training the basic feature extraction model to be trained by using a target function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample and the relation mapping sample.
In one embodiment, the model training module is further configured to obtain an entity description text sample; inputting the entity description text sample into a semantic feature extraction model to be trained, so that the semantic feature extraction model to be trained outputs a head entity semantic feature sample, a head entity semantic mapping sample, a tail entity semantic feature sample and a tail entity semantic mapping sample; and training the basic feature extraction model to be trained and the semantic feature extraction model to be trained by using an objective function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample, the relation mapping sample, the head entity semantic feature sample, the tail entity semantic feature sample and the tail entity semantic mapping sample.
In one embodiment, the model training module is further configured to obtain a positive knowledge-graph sample; carrying out random replacement processing on any element in the triple in the knowledge graph positive sample to construct a knowledge graph negative sample; and taking the knowledge graph positive sample and the knowledge graph negative sample as the knowledge graph samples.
For specific limitations of the knowledge-graph embedding device, reference may be made to the above limitations of the knowledge-graph embedding method, which are not described herein again. The modules in the knowledge-graph embedding apparatus can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 3. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing data such as knowledge maps, entity description texts, entity structure characteristics, relationship characteristics, entity semantic characteristics and the like. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of knowledge-graph embedding.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.
In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method of knowledge-graph embedding, the method comprising:
acquiring entity structural features and relationship features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features;
acquiring an entity description text of the knowledge graph, and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
and embedding the knowledge map by using the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
2. The method according to claim 1, wherein the processing the entity description text by using the pre-constructed semantic feature extraction model to obtain entity semantic features comprises:
segmenting words of the entity description text and expressing the word vectors to obtain a first word vector sequence corresponding to the entity description text;
inputting the first word vector sequence into the semantic feature extraction model, so that the semantic feature extraction model obtains semantic relevancy between each entity vector contained in the first word vector sequence and other word vectors, obtains a second word vector sequence corresponding to the entity description text according to the semantic relevancy, each entity vector and other word vectors, and obtains the entity semantic features according to the second word vector sequence.
3. The method of claim 2, wherein the semantic feature extraction model comprises a self-attention model and a convolutional neural network; wherein the content of the first and second substances,
the self-attention model is used for acquiring semantic relevance of each entity vector contained in the first word vector sequence and other word vectors, and obtaining a second word vector sequence corresponding to the entity description text according to the semantic relevance, each entity vector and other word vectors;
and the convolutional neural network is used for acquiring the entity semantic features according to the second word vector sequence obtained by the self-attention model.
4. The method of claim 1, wherein the base feature extraction model is a transD model.
5. The method according to any one of claims 1 to 4, further comprising:
acquiring a knowledge graph sample;
inputting the knowledge graph sample into a basic feature extraction model to be trained, so that the basic feature extraction model to be trained outputs a head entity structure feature sample, a head entity structure mapping sample, a tail entity structure feature sample, a tail entity structure mapping sample, a relation feature sample and a relation mapping sample;
and training the basic feature extraction model to be trained by using a target function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample and the relation mapping sample.
6. The method of claim 5,
the method further comprises the following steps:
acquiring an entity description text sample;
inputting the entity description text sample into a semantic feature extraction model to be trained, so that the semantic feature extraction model to be trained outputs a head entity semantic feature sample, a head entity semantic mapping sample, a tail entity semantic feature sample and a tail entity semantic mapping sample;
the training of the basic feature extraction model to be trained by using the target function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample and the relation mapping sample comprises the following steps:
and training the basic feature extraction model to be trained and the semantic feature extraction model to be trained by using an objective function constructed based on the head entity structure feature sample, the head entity structure mapping sample, the tail entity structure feature sample, the tail entity structure mapping sample, the relation feature sample, the relation mapping sample, the head entity semantic feature sample, the tail entity semantic feature sample and the tail entity semantic mapping sample.
7. The method of claim 5, wherein obtaining the knowledge-graph sample comprises:
acquiring a positive knowledge graph sample;
carrying out random replacement processing on any element in the triple in the knowledge graph positive sample to construct a knowledge graph negative sample;
and taking the knowledge graph positive sample and the knowledge graph negative sample as the knowledge graph samples.
8. A knowledge-graph embedding apparatus, comprising:
the first feature extraction module is used for acquiring the entity structure features and the relation features of the knowledge graph by using a pre-constructed basic feature extraction model; the solid structure features at least comprise head solid structure features and tail solid structure features;
the second feature extraction module is used for acquiring an entity description text of the knowledge graph and processing the entity description text by utilizing a pre-constructed semantic feature extraction model to obtain entity semantic features; the entity semantic features at least comprise head entity semantic features and tail entity semantic features;
and the embedding processing module is used for embedding the knowledge graph by utilizing the head entity structural feature, the tail entity structural feature, the relation feature, the entity semantic feature and the tail entity semantic feature.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912715.4A CN113626610A (en) | 2021-08-10 | 2021-08-10 | Knowledge graph embedding method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110912715.4A CN113626610A (en) | 2021-08-10 | 2021-08-10 | Knowledge graph embedding method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113626610A true CN113626610A (en) | 2021-11-09 |
Family
ID=78383889
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110912715.4A Pending CN113626610A (en) | 2021-08-10 | 2021-08-10 | Knowledge graph embedding method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626610A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626609A (en) * | 2021-08-10 | 2021-11-09 | 南方电网数字电网研究院有限公司 | Electric power measurement knowledge map construction method, device, equipment and storage medium |
| CN114329234A (en) * | 2022-03-04 | 2022-04-12 | 深圳佑驾创新科技有限公司 | Collaborative filtering recommendation method and system based on knowledge graph |
| CN117610562A (en) * | 2024-01-23 | 2024-02-27 | 中国科学技术大学 | Relation extraction method combining combined category grammar and multi-task learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334219A (en) * | 2019-07-12 | 2019-10-15 | 电子科技大学 | The knowledge mapping for incorporating text semantic feature based on attention mechanism indicates learning method |
| CN111198950A (en) * | 2019-12-24 | 2020-05-26 | 浙江工业大学 | Knowledge graph representation learning method based on semantic vector |
| CN112000815A (en) * | 2020-10-28 | 2020-11-27 | 科大讯飞(苏州)科技有限公司 | Knowledge graph complementing method and device, electronic equipment and storage medium |
| CN112131404A (en) * | 2020-09-19 | 2020-12-25 | 哈尔滨工程大学 | Entity alignment method in four-risk one-gold domain knowledge graph |
| CN113032582A (en) * | 2021-04-20 | 2021-06-25 | 杭州叙简科技股份有限公司 | Knowledge graph based entity unified model establishment and entity unified method |
-
2021
- 2021-08-10 CN CN202110912715.4A patent/CN113626610A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334219A (en) * | 2019-07-12 | 2019-10-15 | 电子科技大学 | The knowledge mapping for incorporating text semantic feature based on attention mechanism indicates learning method |
| CN111198950A (en) * | 2019-12-24 | 2020-05-26 | 浙江工业大学 | Knowledge graph representation learning method based on semantic vector |
| CN112131404A (en) * | 2020-09-19 | 2020-12-25 | 哈尔滨工程大学 | Entity alignment method in four-risk one-gold domain knowledge graph |
| CN112000815A (en) * | 2020-10-28 | 2020-11-27 | 科大讯飞(苏州)科技有限公司 | Knowledge graph complementing method and device, electronic equipment and storage medium |
| CN113032582A (en) * | 2021-04-20 | 2021-06-25 | 杭州叙简科技股份有限公司 | Knowledge graph based entity unified model establishment and entity unified method |
Non-Patent Citations (2)
| Title |
|---|
| RUOBING XIE: "Representation Learning of Knowledge Graphs with Entity Descriptions", 《THIRTIETH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE》, pages 2659 - 2665 * |
| 谭真: "面向非结构化数据的知识图谱构建与表示技术研究", 《中国博士学位论文全文数据库 信息科技》, pages 138 - 283 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626609A (en) * | 2021-08-10 | 2021-11-09 | 南方电网数字电网研究院有限公司 | Electric power measurement knowledge map construction method, device, equipment and storage medium |
| CN113626609B (en) * | 2021-08-10 | 2024-03-26 | 南方电网数字电网研究院有限公司 | Electric power metering knowledge graph construction method, device, equipment and storage medium |
| CN114329234A (en) * | 2022-03-04 | 2022-04-12 | 深圳佑驾创新科技有限公司 | Collaborative filtering recommendation method and system based on knowledge graph |
| CN117610562A (en) * | 2024-01-23 | 2024-02-27 | 中国科学技术大学 | Relation extraction method combining combined category grammar and multi-task learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368996B (en) | Retraining projection network capable of transmitting natural language representation | |
| CN113626610A (en) | Knowledge graph embedding method and device, computer equipment and storage medium | |
| CN112084331A (en) | Text processing method, text processing device, model training method, model training device, computer equipment and storage medium | |
| CN108288067A (en) | Training method, bidirectional research method and the relevant apparatus of image text Matching Model | |
| CN113593611B (en) | Voice classification network training method and device, computing equipment and storage medium | |
| CN113420163B (en) | Heterogeneous information network knowledge graph completion method and device based on matrix fusion | |
| US20170116521A1 (en) | Tag processing method and device | |
| CN113268612B (en) | Heterogeneous information network knowledge graph completion method and device based on mean value fusion | |
| KR102250728B1 (en) | Sample processing method and device, related apparatus and storage medium | |
| CN111782826A (en) | Knowledge graph information processing method, device, equipment and storage medium | |
| CN111311107A (en) | Risk assessment method and device based on user relationship and computer equipment | |
| US20190228297A1 (en) | Artificial Intelligence Modelling Engine | |
| CN109447129B (en) | Multi-mode emotion recognition method and device and computer readable storage medium | |
| CN111597352B (en) | Network space knowledge graph reasoning method and device combining ontology concepts and instances | |
| Liu et al. | EACP: An effective automatic channel pruning for neural networks | |
| CN113254649B (en) | Training method of sensitive content recognition model, text recognition method and related device | |
| CN111783688B (en) | Remote sensing image scene classification method based on convolutional neural network | |
| CN113326940A (en) | Knowledge distillation method, device, equipment and medium based on multiple knowledge migration | |
| CN116992151A (en) | Online course recommendation method based on double-tower graph convolution neural network | |
| CN109977194B (en) | Text similarity calculation method, system, device and medium based on unsupervised learning | |
| CN116957006A (en) | Training method, device, equipment, medium and program product of prediction model | |
| CN113590720A (en) | Data classification method and device, computer equipment and storage medium | |
| CN112650861A (en) | Personality prediction method, system and device based on task layering | |
| CN112149389A (en) | Resume information structured processing method and device, computer equipment and storage medium | |
| CN111159450A (en) | Picture classification method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |