CN112632291B - Generalized atlas characterization method with enhanced ontology concept - Google Patents
Generalized atlas characterization method with enhanced ontology concept Download PDFInfo
- Publication number
- CN112632291B CN112632291B CN202011534627.7A CN202011534627A CN112632291B CN 112632291 B CN112632291 B CN 112632291B CN 202011534627 A CN202011534627 A CN 202011534627A CN 112632291 B CN112632291 B CN 112632291B
- Authority
- CN
- China
- Prior art keywords
- entity
- instance entity
- concept
- instance
- ontology
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012512 characterization method Methods 0.000 title claims description 64
- 239000013598 vector Substances 0.000 claims abstract description 55
- 230000007246 mechanism Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims description 25
- 230000004931 aggregating effect Effects 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 2
- 238000013519 translation Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 6
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 239000000523 sample Substances 0.000 description 15
- 230000006870 function Effects 0.000 description 11
- 239000010410 layer Substances 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000001939 inductive effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000001052 transient effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 229910052573 porcelain Inorganic materials 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a generalized map representation method for enhancing ontology concepts, which can generate ontology concept representations containing rich information through a double-layer attention mechanism and can be used for effectively improving the embedding effect of newly added entities; the attention mechanism determined by the relation is used for fusing a plurality of concepts corresponding to a newly added entity to generate an entity template vector; the template vector is further fused with personalized features provided by neighbors in the newly added entity to generate a newly added entity vector representation, and finally, the effect of the map completion task is effectively improved.
Description
Technical Field
The invention relates to the field of knowledge graph representation learning in natural language processing, in particular to a generalized graph representation method for enhancing ontology concepts.
Background
The knowledge graph contains a plurality of instance entity triples, wherein the instance entity triples can be expressed in the form of (head entity, relation and tail entity), and one instance entity triplet represents a piece of knowledge. The knowledge graph plays an increasingly important role in tasks such as information retrieval, question answering, recommendation and the like, and the application range of the knowledge graph is also continuously expanded. However, existing knowledge spectrograms generally have the problem of being not complete enough, namely the relationship between a large number of entities is still not contained in the spectrograms. The graph completion task based on representation learning aims at learning vector representations of entities and relationships, and further predicts missing relationships between entities based on the entities and the relationship vectors. Conventional direct-push representation learning methods assume that all test entities are visible during the training phase. However, in real-world scenarios, the atlas is still refined after construction, which results in the continued appearance of new entities in the atlas. The direct-push representation learning method has to retrain the whole map to obtain the representation of the newly added entity, which is very inefficient and resource-consuming.
Therefore, the generalized graph representation learning method aims at generating the representation of the newly added entity in a generalized way, so that resources are saved, and the requirement of real-time calculation is met. At present, a few related technical schemes and research results exist for the generalized expression learning method of the newly added entity, and part of representative disclosure technologies include: CN202010809387.0, a local training method for knowledge-graph representation learning, uses vector representation of original entity and relation of graph to obtain initialization representation of new entity according to the transition model, and then carries out fine tuning. CN201911380039.X, a knowledge graph representation learning method based on anchor points, which uses text information as a semantic basis of a newly added entity and combines relevant local knowledge of the existing knowledge graph for training.
The prior art can be divided into two categories: (1) Based on the method of the internal neighbors of the newly added entity (such as patent CN 202010809387.0), the method generally uses a graph convolution neural network model to fuse the internal neighbors of the newly added entity, so as to inductively generate the representation of the newly added entity. (2) Based on the method of the description information of the newly added entity (for example, patent cn201911380039. X), such methods generally use the text or image description information of the newly added entity, and use a text or image embedding tool to obtain the vector representation of the newly added entity.
However, for the method in (1), the representation of the newly added entity generated by a simple fusion algorithm is often not accurate enough due to the sparsity and heterogeneity of the neighbors inside the newly added entity. For the method in (2), the effect of the characterization is highly dependent on the quality of the descriptive information. In practical applications, it is difficult to obtain high quality descriptive information that meets the requirements.
Disclosure of Invention
The invention aims to provide a body concept enhanced inductive spectrum characterization method which inductively generates characterization of newly added entities, so that the characterization of the newly added entities is more accurate and efficient, and the accuracy of a downstream spectrum completion task is improved.
The invention aims at realizing the following technical scheme:
a method of generalized atlas characterization with enhanced ontology concepts, comprising:
constructing a network model, and giving a triplet containing the newly added instance entity, and giving the ontology concept and the neighbor instance entity set of each newly added instance entity in the triplet; the network model generates representation of the ontology concept through a double-layer attention mechanism for each ontology concept of each newly added instance entity in the triplet; generating template characterization of the newly added instance entity based on the characterization of all ontology concepts and the triples containing the newly added instance entity, and generating a final characterization vector of the newly added instance entity by combining with the neighbor instance entity set; evaluating the legitimacy of the triples containing the newly added instance entities based on the final characterization vectors of all the newly added instance entities; if the legal requirements are met, the triples are added to the knowledge graph.
According to the technical scheme provided by the invention, the ontology concept representation containing rich information can be generated through a double-layer attention mechanism, and the ontology concept representation can be used for effectively improving the embedding effect of the newly added entity; the attention mechanism determined by the relation is used for fusing a plurality of concepts corresponding to a newly added entity to generate an entity template vector; the template vector is further fused with personalized features provided by neighbors in the newly added entity to generate a newly added entity vector representation, and finally, the effect of the map completion task is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for representing a generalized atlas with enhanced ontology concept according to an embodiment of the present invention;
fig. 2 is a model diagram of a generalized graph characterization method with enhanced ontology concept according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
Both types of methods in the prior art ignore an information important to the newly added entity, namely, the ontology concept. The knowledge graph comprises instance entities and corresponding ontology concepts. In one aspect, instance entities provide rich detailed information for corresponding ontology concepts. In another approach, the ontology concept provides a basic summary information for its entity, which is particularly important for the newly added entity. The ontology concept can be used as a basic template of the newly added entity to provide a relatively accurate position range in the vector space, so that the embodiment of the invention provides a generalized graph characterization method with enhanced ontology concept, which mainly comprises the following steps as shown in fig. 1:
constructing a network model as shown in fig. 2, and giving a triplet containing newly added instance entities, and simultaneously giving an ontology concept and a neighbor instance entity set of each newly added instance entity in the triplet; the network model generates representation of the ontology concept through a double-layer attention mechanism for each ontology concept of each newly added instance entity in the triplet; generating template characterization of the newly added instance entity based on the characterization of all ontology concepts and the triples containing the newly added instance entity, and generating a final characterization vector of the newly added instance entity by combining with the neighbor instance entity set; evaluating the legitimacy of the triples containing the newly added instance entities based on the final characterization vectors of all the newly added instance entities; if the legal requirements are met, the triples are added into the knowledge graph, and the integrity of the graph is improved.
The above description is presented as the main principle of the whole scheme, and the network model needs training and parameter estimation and then is used for prediction tasks.
For ease of understanding, the following description is made in connection with the process of model training and parameter estimation, and the task of prediction, in conjunction with the principles described above.
1. And (5) finishing and preprocessing basic data.
Before model training and parameter estimation, basic data needs to be collected and arranged, and preprocessing of related data is performed, wherein the following preferred embodiment is as follows:
1. and (5) arranging basic data.
In the embodiment of the invention, the basic data is a knowledge graph containing body concept information, and mainly comprises three parts of data information: an instance entity triplet representing a relationship between instance entities; an ontology concept triplet representing a meta-relationship between ontology concepts; example entity concept pairs represent the correspondence between example entities and their own ontology concepts, and the data are usually in text form.
In the embodiment of the invention, the meta-relationship among the ontology concepts can reflect the association relationship among different ontology concepts, such as (city at_location state), and for a special meta-relationship subs_of (city subs_of place), the parent-child relationship among the ontology concepts is reflected, and other meta-relationships (non-subs_of) are general meta-relationships. For a concept such as the city, according to the relationship of the subs_of, the parent concept and the child concept of the concept can be obtained in an arrangement mode. From other general meta-relationships (e.g., at_location), general neighbor concepts (e.g., state) can be derived.
To facilitate model training, the atlases need to be sorted. After finishing, obtaining a neighbor instance entity set and a corresponding ontology concept set for each instance entity; for each ontology concept, the child concept sets, the father concept sets, the general adjacent concept sets and the corresponding instance entity sets are obtained by sorting.
2. And (5) preprocessing data.
In the embodiment of the invention, the data preprocessing corresponds to an instance entity triplet, and a negative sample is constructed through preprocessing to perform model training. And taking the instance entity triplet in the knowledge graph as positive samples, generating negative samples by randomly replacing a head entity or a tail entity in the instance entity triplet for each positive sample to form positive and negative sample pairs, and carrying out model training and parameter estimation by combining a plurality of positive and negative sample pairs.
In the embodiment of the invention, for a positive triplet, only one of the head instance entity or the tail instance entity is replaced, and the instance entity for replacement is randomly selected in the map.
2. Model training and parameter estimation.
Model training and parameter estimation mainly comprises three parts:
the first part is to compute a representation of the ontology concept: recording any instance entity (including a head instance entity and a tail instance entity) in a positive and negative sample pair as a target instance entity, extracting a corresponding set of ontology concepts, modeling a local hierarchy using a dual-layer attention mechanism for each ontology concept, the ontology concepts being associated with multiple types of nodes, the nodes comprising: other ontology concepts and instance entities; firstly, for each type of node, using a node level attention mechanism to fuse node information of the same type, thereby obtaining information characterization of each node type; the information tokens of each type are then aggregated using a type-level attention mechanism, resulting in tokens of the ontology concept.
The second part is to calculate the final token vector for the target instance entity: according to the relation information in the instance entity triples, a relation-determined attention mechanism is used for generating a template representation of the target instance entity in combination with representations of all ontology concepts corresponding to the target instance entity, and a gate mechanism is adopted for generating a final representation vector of the target instance entity in combination with the template representation of the target instance entity and a neighbor instance entity set of the target instance entity.
The third part is to calculate the loss by scoring: and calculating scores of the positive sample and the negative sample by using a scoring function and combining final characterization vectors of all instance entities in the positive sample and the negative sample, and constructing a loss function by using the scoring result to perform parameter estimation of the model.
The preferred embodiments of the three parts are as follows:
1. a representation of the ontology concept is calculated.
In the embodiment of the invention, the local hierarchical structure of the ontology concept is modeled by using a double-layer attention mechanism, the local hierarchical structure information of the ontology concept is fully considered, and the representation of the ontology concept is obtained by modeling. Consider that each ontology concept is typically associated with four types of information: parent concepts, child concepts, general neighbor concepts, and instance entities; therefore, for each type of node (father, son, general adjacent concepts are ontology concepts, example entity is not concept, for one ontology concept c, it has the four types of neighbor nodes, in the knowledge graph, the concepts, entities are nodes, the relationship is edges, only the types are different), using the node level attention mechanism, fusing the same type of node set, thus obtaining each type of information characterization; the information tokens of each type are then aggregated using a type-level attention mechanism, resulting in tokens of the ontology concept.
In the embodiment of the invention, a node level attention mechanism is provided to represent that the weights of adjacent nodes under the same type are different in terms of representation target concept. For example, in an example entity of the concept of "singer", a top singer such as "Zhou Jielun" may be more representative than other singers. Thus, the same type of node is taken as a groupFor each node in the group, its weight is derived using a node level attention mechanism, the calculation process is as follows:
wherein t epsilon {1,2,3,4} represents 4 groups of different types of neighbor node information, which are respectively a father concept set, a son concept set, a general adjacent concept set and an instance entity set corresponding to the ontology concept obtained in the knowledge graph arrangement stage;representing the original representation of the ontology concept c, +.>D is the dimension of the token vector, which is the real set; />Is the ith node of type t +.>Is characterized by vector>A node set of type t representing the ontology concept c; the symbol || represents a stitching operation, and sigma is a LeakyReLU function; />And->Is a training parameter related to t.
When (when)Representing a parent concept, or a child concept, or an instance entity +.>Representing a corresponding concept representation or entity representation; but when->Representing a generalThere is also an associated meta-relationship when adjacent concepts. The present invention uses the basic assumption of TransE, employing the transformation +.>To express the influence of the meta-relationship, here +.>Representing general neighboring concept token vectors,/->Representing the meta-relationship token vector. The characterization vectors of all the entities, concepts and relations are obtained through random initialization and are continuously adjusted as parameters in the training process.
The node level attention value is calculated by the above operationAnd fusing neighbor nodes under the same type to obtain information characterization of each type, wherein the information characterization is expressed as follows:
after each type of information representation is obtained, the embodiment of the invention provides a type-level attention mechanism to fuse each type of representation, so as to obtain a final concept representation containing rich information. The type-level attention mechanism takes into account that different types of information have different effects on representing the target concept, and for the ontology concept c, the type-level attention value of each type is calculated using the type-level attention mechanismExpressed as:
wherein,and->Is a training parameter.
Finally, aggregating the information characterizations of each type to obtain a characterization of the ontology concept, which is expressed as:
where c' represents a characterization of the ontology concept c by a double-layer attention mechanism.
The type-level attention values calculated by the embodiment of the invention are implicitly shared by nodes of the same type, which can promote information sharing between the nodes. In fact, the two-layer attentiveness mechanism provides a finer granularity process for learning attentiveness values while improving the interpretability of the model to some extent.
2. A final token vector for the target instance entity is calculated.
In the first part, modeling of the local two-level hierarchical structure of the ontology concept is completed, and concept characterization containing rich information is obtained. This section will design a generalized entity characterization scheme based on the characterization of the ontology concept.
The prior structure-based inductive entity characterization method obtains the characterization of the target instance entity by aggregating neighbor instance entities of the target instance entity. This approach has limited effectiveness in modeling when dealing with newly added entities. Because neighbor instance entities of the newly added instance entity are typically very sparse, and in the knowledge graph, the instance entities are heterogeneous. In order to solve the problem, the invention provides a template perfecting strategy to express target entities (applicable to newly added instance entities) in a generalized way. Here, for one target instance entity, its corresponding ontology concept is used in addition to its limited heterogeneous neighbor entities. In embodiments of the present invention, the ontology concept may be considered to describe the basic outline of the current instance entity, while the neighbor instance entities of the target instance entity may provide personalized features for it. For example, for a target instance entity, given its ontology concept "singer", the approximate location of that instance entity in vector space can be known. Meanwhile, given its neighbor instance entity: (e.g., "stands for", "blue and white porcelain") may be physically distinguished from other instances that are also "singers".
Considering that a target instance entity may correspond to multiple ontology concepts, such as for an "adult" instance entity, the corresponding ontology concepts are: "actor", "husband", etc. However, for a given example entity triplet ("Dragon", "lead", "A plan"), the ontology concept "actor" is more important here, as can be known from the relationship "lead".
Therefore, in the embodiment of the invention, the influence of different concepts is measured by using a relation-determined attention mechanism, so that the template characterization vector of the target entity is generated. First, an attention value gamma is calculated using an attention mechanism determined by a relationship i Expressed as:
wherein c' i The ith ontology concept c, which is the target instance entity i R is a representation vector of a relationship in an instance entity triplet to which the target instance entity belongs;is a set of ontology concepts corresponding to the target instance entities; />And->Is a training parameter.
Then, a template representation of the target instance entity is computed, expressed as:
wherein t is e For template characterization of the target instance entity, it describes summary information of the target instance entity. With this as a reference, the characterization of the target instance entity obtained will not have excessive errors. The neighbor instance entities of the target instance entity will then be provided with personalized information, thereby obtaining a more accurate representation of the target instance entity. In embodiments of the present invention, each neighbor entity is considered to exhibit a characteristic unique to certain aspects of the target entity, which may be formalized as a scaling of the template representation in certain dimensions determined by the neighbor entity. Thus, we employ a gating mechanism so that each neighbor entity can weight all dimensions of the template, which can be described by:
n i ＝h i +r i
y i ＝tanh(Ut e +Vn i )
wherein n is i The ith neighbor instance entity n, which is the target instance entity i ，And->Adjacent head entity representation vector and relation representation vector, respectively,/->Representing a set of neighbor instance entities, +.>Representation set->Element number of->And->Is a training parameter, e' represents the final token vector for the target instance entity.
3. The TransE was used as a scoring function and the loss was calculated.
In the embodiment of the invention, the transient is used as a scoring function, and for positive samples (h, r, t), the basic idea of the transient is to consider the relation r as a translation from a head instance entity h to a tail instance entity t, and the corresponding representation vector form is h+r apprxeq t; h and t each represent a final characterization vector calculated by using the above scheme by taking a head instance entity h and a tail instance entity t as target instance entities, and r is a characterization vector of a relation r (which can be obtained by a conventional manner); the scoring function is expressed as:
f(h，r，t)＝||h+r-t||
wherein, the score function can estimate the probability that the instance entity triplet is a positive sample, i.e. the validity of the triplet, and for legal triples, the smaller the f-value, the better.
For positive samples (h, r, t), the negative samples obtained by the previous pretreatment are noted as (h ', r, t'), and the scores are calculated in the same way:
f(h′，r，t′)＝||h′+r-t′||
only one of h ', t' in the negative sample is replaced, the other being identical to the instance entity in the positive sample; h 'and t' are also calculated according to the schemes provided above.
Constructing a loss function according to the evaluation result of each positive and negative sample pair, wherein the loss function is expressed as follows:
wherein N is the number of positive and negative sample pairs, τ is a superparameter, [ x ]] + Representation function [ x ]] + =ma× (0, x). The loss function described above treats the positive and negative samples independently, the fraction of positive samples will tend to be 0, and the fraction of negative triples will be greater than or equal to τ.
In the model training and parameter estimation, based on the trained parameters, the influence of the adjacent nodes on the target concept representation can be measured in two levels (node level and type level) for the ontology concept; for the target entity, the basic information of the target entity can be obtained based on the ontology concept, and the personalized characteristics of the target entity can be obtained through the perfection of the neighbor entity. The thus obtained target instance entity characterization vector will be more accurate.
3. The model is applied to the predictive task.
After the network model is trained, a triplet containing the newly added instance entity is given, meanwhile, the ontology concept and the neighbor instance entity set of the newly added instance entity are given, the trained network model is subjected to inductive calculation to obtain the final characterization vector of the newly added instance entity, the legality of the given triplet is evaluated, and the calculation details related to the stage are the same as the model training and parameter estimation, so that the details are not repeated.
In the embodiment of the invention, in the triples containing the newly added instance entities, the head instance entity h, the tail instance entity t and the relations of the head instance entity h, the tail instance entity t are newly added, and meanwhile, the triples are not contained in the original knowledge graph, and the characterization vectors of the head instance entity and the tail instance entity are calculated in a inductive manner through the scheme, so that when an evaluation result shows that the ternary combination method, the ternary combination method can be added to the knowledge graph to improve the integrity of the graph.
From the description of the above embodiments, it will be apparent to those skilled in the art that the above embodiments may be implemented in software, or may be implemented by means of software plus a necessary general hardware platform. With such understanding, the technical solutions of the foregoing embodiments may be embodied in a software product, where the software product may be stored in a nonvolatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.), and include several instructions for causing a computer device (may be a personal computer, a server, or a network device, etc.) to perform the methods of the embodiments of the present invention.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (9)
1. A method for generalized atlas characterization with enhanced ontology concepts, comprising:
constructing a network model, and giving a triplet containing the newly added instance entity, and giving the ontology concept and the neighbor instance entity set of each newly added instance entity in the triplet; the network model generates representation of the ontology concept through a double-layer attention mechanism for each ontology concept of each newly added instance entity in the triplet; generating template characterization of the newly added instance entity based on the characterization of all ontology concepts and the triples containing the newly added instance entity, and generating a final characterization vector of the newly added instance entity by combining with the neighbor instance entity set; evaluating the legitimacy of the triples containing the newly added instance entities based on the final characterization vectors of all the newly added instance entities; if the legal requirements are met, adding the triples into the knowledge graph;
in the training stage of the network model, basic data are collected and arranged, wherein the basic data are knowledge graphs containing body concept information, and the knowledge graphs contain three parts of data information: an instance entity triplet representing a relationship between instance entities; an ontology concept triplet representing a meta-relationship between ontology concepts; an instance entity concept pair, which represents the corresponding relation between an instance entity and the entity concept to which the instance entity belongs, wherein the data is in a text form; after that, pretreatment is performed: and taking the instance entity triplet in the knowledge graph as a positive sample, generating a negative sample by adopting a mode of randomly replacing a head entity or a tail entity in the instance entity triplet to form a positive and negative sample pair, and carrying out model training and parameter estimation by combining a plurality of positive and negative sample pairs.
2. The method for generalized atlas characterization with enhanced ontology concepts according to claim 1, wherein the model training and parameter estimation performed in combination with a plurality of positive and negative sample pairs, includes:
a first part: recording any instance entity in positive and negative sample pairs as a target instance entity, extracting a corresponding set of ontology concepts, and modeling a local hierarchy structure by using a double-layer attention mechanism for each ontology concept, wherein the ontology concepts are associated with a plurality of types of nodes, and the nodes comprise: other ontology concepts and instance entities; firstly, for each type of node, using a node level attention mechanism to fuse node information of the same type, thereby obtaining information characterization of each node type; then, using a type-level attention mechanism to aggregate information characterization of each type to obtain characterization of the ontology concept;
a second part: according to relation information in the instance entity triples, a relation-determined attention mechanism is used for generating template characterization of the target instance entity in combination with characterization of all ontology concepts corresponding to the target instance entity, and a gate mechanism is adopted for generating a final characterization vector of the target instance entity in combination with the template characterization of the target instance entity and a neighbor instance entity set of the target instance entity;
third section: and calculating scores of the positive sample and the negative sample by using a scoring function and combining final characterization vectors of all instance entities in the positive sample and the negative sample, and constructing a loss function by using the scoring result to perform parameter estimation of the model.
3. The method for representing the generalized atlas with enhanced ontology concept according to claim 2, wherein after the knowledge atlas is sorted, each instance entity is obtained from a neighboring instance entity set and a corresponding ontology concept set; for each ontology concept, the child concept sets, the father concept sets, the general adjacent concept sets and the corresponding instance entity sets are obtained by sorting.
4. The method of claim 2, wherein, in the first portion, for each ontology concept, using a node level attention mechanism, fusing concept information of the same type, thereby obtaining information tokens of respective types comprises:
the same type of node being a groupFor each node in the group, a node level attention value of each node in the group is derived using a node level attention mechanism>Expressed as:
wherein t epsilon {1,2,3,4} represents 4 groups of node information of different types, which are respectively a father concept set, a son concept set, a general adjacent concept set and an instance entity set corresponding to the ontology concept obtained in the knowledge graph arrangement stage;representing the original representation of the ontology concept c, +.>D is the dimension of the token vector, which is the real set; />Is a token vector of the ith node of type t,/->A node set of type t representing the ontology concept c; the symbol || represents a stitching operation, and sigma is a LeakyReLU function;and->Is a training parameter related to t;
and fusing node information under the same type to obtain information characterization of each node type, wherein the information characterization is expressed as follows:
5. the method of claim 2 or 4, wherein aggregating information tokens of respective types using a type-level attention mechanism to obtain tokens of an ontology concept comprises:
computing type-level attention values for each type using a type-level attention mechanismExpressed as:
wherein t epsilon {1,2,3,4} represents 4 groups of node information of different types, which are respectively a father concept set, a son concept set, a general adjacent concept set and an instance entity set corresponding to the ontology concept obtained in the knowledge graph arrangement stage; m is m t Representing information characterization with the type t;representing the original representation of the ontology concept c, +.>And->Is a training parameter->D is the dimension of the token vector, which is the real set;
and then aggregating the information characterizations of each type to obtain the characterization of the ontology concept, which is expressed as follows:
where c' represents a characterization of the ontology concept c by a double-layer attention mechanism.
6. The method of claim 2, wherein in the second portion, using the attention mechanism of the relationship determination to generate the template representation of the target instance entity in combination with the representations of all the ontology concepts corresponding to the target instance entity comprises:
first, an attention value gamma is calculated using an attention mechanism determined by a relationship i Expressed as:
wherein c' i The ith ontology concept c, which is the target instance entity i R is a representation vector of a relationship in an instance entity triplet to which the target instance entity belongs;is a set of ontology concepts corresponding to the target instance entities; /> And->Is a training parameter->D is the dimension of the token vector, which is the real set;
then, a template representation of the target instance entity is computed, expressed as:
wherein t is e Template characterization for the target instance entity.
7. The method for representing a generalized atlas with enhanced ontology concept according to claim 2 or 6, wherein a gate mechanism is adopted to combine template representation of the target instance entity and a neighbor instance entity set of the target instance entity to generate a final representation vector of the target instance entity, which is expressed as:
n i ＝h i +r i
y i ＝tanh(Ut e +Vn i )
wherein n is i The ith neighbor instance entity n, which is the target instance entity i ，And->Adjacent head entity representation vector and relation representation vector, respectively,/->Representing a set of neighbor instance entities, +.>Representation set->Element number of->And->Is a training parameter->Is realSet, d is the dimension characterizing the vector, t e For template characterization of the target instance entity, e' represents the final characterization vector of the target instance entity.
8. The method of claim 2, wherein in the third section, using the transition as a scoring function, for positive samples (h, r, t), the relation r is considered as a translation from the head instance entity h to the tail instance entity t, and the corresponding token vector is of the form h+r≡t; the scoring function is expressed as:
f(h,r,t)＝||h+r-t||
wherein, I represents I1 or l2 norm; h and t respectively represent final characterization vectors of a head instance entity h and a tail instance entity t in the positive sample; r is a characterization vector of the relation r;
for the negative samples (h ', r, t'), scores were calculated in the same manner,
f(h′,r,t′)＝||h′+r-t′||
only one of h ', t' in the negative sample is replaced, the other being identical to the instance entity in the positive sample; h 'and t' each represent the final token vector of the instance entity h 'and the tail instance entity t' in the negative sample;
the final loss function is:
wherein N is the number of positive and negative sample pairs, τ is a superparameter, [ x ]] + Representation function [ x ]] + ＝max(0,x)。
9. The method for representing the generalized atlas with enhanced ontology concept according to claim 2, wherein after the network model is trained, a triplet including the newly added instance entity is given, and the ontology concept and the neighbor instance entity set of the newly added instance entity are given at the same time, so that the final representation vector and the validity evaluation result of the newly added instance entity can be obtained by using the trained network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011534627.7A CN112632291B (en) | 2020-12-23 | 2020-12-23 | Generalized atlas characterization method with enhanced ontology concept |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011534627.7A CN112632291B (en) | 2020-12-23 | 2020-12-23 | Generalized atlas characterization method with enhanced ontology concept |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112632291A CN112632291A (en) | 2021-04-09 |
| CN112632291B true CN112632291B (en) | 2024-02-23 |
Family
ID=75321320
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011534627.7A Active CN112632291B (en) | 2020-12-23 | 2020-12-23 | Generalized atlas characterization method with enhanced ontology concept |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112632291B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851613A (en) * | 2019-09-09 | 2020-02-28 | 中国电子科技集团公司电子科学研究院 | Method and device for complementing, deducing and storing knowledge graph based on entity concept |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2003077079A2 (en) * | 2002-03-08 | 2003-09-18 | Enleague Systems, Inc | Methods and systems for modeling and using computer resources over a heterogeneous distributed network using semantic ontologies |
-
2020
- 2020-12-23 CN CN202011534627.7A patent/CN112632291B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851613A (en) * | 2019-09-09 | 2020-02-28 | 中国电子科技集团公司电子科学研究院 | Method and device for complementing, deducing and storing knowledge graph based on entity concept |
Non-Patent Citations (1)
| Title |
|---|
| 基于概念图谱与BiGRU-Att模型的突发事件演化关系抽取;余蓓;刘宇;顾进广;;武汉科技大学学报(02);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112632291A (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3467723B1 (en) | Machine learning based network model construction method and apparatus | |
| CN112529168B (en) | GCN-based attribute multilayer network representation learning method | |
| Huang et al. | Pattern trees induction: A new machine learning method | |
| CN111191709B (en) | Continuous learning framework and continuous learning method of deep neural network | |
| CN114048331A (en) | Knowledge graph recommendation method and system based on improved KGAT model | |
| Ulhaq et al. | Efficient diffusion models for vision: A survey | |
| CN111126218A (en) | Human behavior recognition method based on zero sample learning | |
| CN112418525B (en) | Method and device for predicting social topic group behaviors and computer storage medium | |
| CN109948680A (en) | The classification method and system of medical record data | |
| CN114511737B (en) | Training method of image recognition domain generalization model | |
| Wang et al. | A modified generative adversarial network for fault diagnosis in high-speed train components with imbalanced and heterogeneous monitoring data | |
| CN113221950A (en) | Graph clustering method and device based on self-supervision graph neural network and storage medium | |
| Zhang et al. | Gaussian metric learning for few-shot uncertain knowledge graph completion | |
| WO2020228536A1 (en) | Icon generation method and apparatus, method for acquiring icon, electronic device, and storage medium | |
| CN113869424A (en) | Semi-supervised node classification method based on two-channel graph convolutional network | |
| Wang et al. | M2SPL: Generative multiview features with adaptive meta-self-paced sampling for class-imbalance learning | |
| Chen et al. | Polydiffuse: Polygonal shape reconstruction via guided set diffusion models | |
| Putra et al. | Multilevel neural network for reducing expected inference time | |
| CN112632291B (en) | Generalized atlas characterization method with enhanced ontology concept | |
| Satapathy et al. | Unsupervised feature selection using rough set and teaching learning-based optimisation | |
| Chen et al. | Wavelet transform-based 3d landscape design and optimization for digital cities | |
| CN114757581A (en) | Financial transaction risk assessment method and device, electronic equipment and computer readable medium | |
| CN114116969A (en) | Corpus screening method based on multiple loss fusion text classification model results | |
| Baldwin et al. | Smoothing the rough edges: Evaluating automatically generated multi-lattice transitions | |
| Zhou et al. | Improved extension neural network and its applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |