CN115169433A - Knowledge graph classification method based on meta-learning and related equipment - Google Patents
Knowledge graph classification method based on meta-learning and related equipment Download PDFInfo
- Publication number
- CN115169433A CN115169433A CN202210610864.XA CN202210610864A CN115169433A CN 115169433 A CN115169433 A CN 115169433A CN 202210610864 A CN202210610864 A CN 202210610864A CN 115169433 A CN115169433 A CN 115169433A
- Authority
- CN
- China
- Prior art keywords
- category
- embedded representation
- training
- embedded
- entity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Animal Behavior & Ethology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a knowledge graph entity classification method based on meta-learning and related equipment. The method comprises the following steps: acquiring an open source knowledge graph data set; initializing the entities and the classes by utilizing a pre-constructed graph neural network model to obtain embedded representation of each entity and embedded representation of each class; obtaining a fused embedded representation of the category by adopting an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view; and inputting the embedded representation of all the entities and the fused embedded representation of all the classes into a classification model, and outputting the class corresponding to each entity through the classification model. The method and the related equipment provided by the application can improve the accuracy of the classification of the knowledge graph entities under the condition that the number of the knowledge graph labeled entities is sparse, and solve the problem of long-tail distribution of the classification tasks of the knowledge graph entities.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a knowledge graph classification method based on meta-learning and related equipment.
Background
Meta Learning (Meta Learning), meaning academic Learning, is intended to enable models to acquire an ability to learn "academic Learning" and to learn new tasks quickly based on existing "knowledge".
The proliferation of the internet has led to an increasing number of large-scale Knowledge bases such as Freebase, google knowledgegrade Graph, YAGO, etc. These large repositories are often composed of an ontology view containing abstract concepts and an instance view composed of entities, and both views contain a large amount of triple relationship data which can provide data support for downstream tasks related to knowledge maps. In the ontology view, the triples comprise a head category, a meta-relation and a tail category, and in the example view, the triples comprise a head entity, a relation and a tail entity. Between the two views, a number of cross-view links are also included to indicate the corresponding categories of entities. The cross-view links can be used as auxiliary information to realize downstream tasks of the knowledge graph, and data bases are laid for tasks such as relation expansion, coreference resolution, entity linking and the like.
However, at the same time, cross-view links themselves present serious problems. For example, in the knowledge base, the number of cross-view links is often small, which affects the performance of the tasks downstream of the knowledge graph, and the knowledge graph entity classification task faces great difficulty, thus seriously hindering the accuracy of the knowledge graph entity classification. In order to solve the problem of knowledge graph entity classification, researchers try to propose various methods, such as completing classification tasks by means of external information in an auxiliary mode, and then completing classification tasks by generating representations of each entity through representation learning methods, but the methods all assume that each class has enough labeled samples, and ignore possible long tail distribution problems of entity classes in a knowledge graph.
Disclosure of Invention
In view of the above, the present application aims to provide a method and related device for knowledge graph classification based on meta-learning.
Based on the above purpose, the present application provides a method for classifying knowledge-graph entities based on meta-learning, comprising:
obtaining an open source knowledge-graph dataset, the open source knowledge-graph dataset comprising an ontology view and an instance view, the ontology view comprising a plurality of categories, the instance view comprising a plurality of entities;
initializing the entities and the categories by utilizing a pre-constructed graph neural network model to obtain embedded representation of each entity and embedded representation of each category;
based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view, obtaining a fused embedded representation of the category by adopting an attention mechanism algorithm;
and inputting the embedded representation of all the entities and the fused embedded representation of all the categories into a classification model, and outputting the category corresponding to each entity through the classification model, wherein the classification model is pre-trained by adopting a meta-learning method.
Based on the same inventive concept, the present disclosure also provides a knowledge graph classification device for meta-learning based on complex contexts, comprising:
an acquisition module that acquires an open source knowledge-graph dataset, the open source knowledge-graph dataset comprising an ontology view and an instance view, the ontology view comprising a plurality of categories, the instance view comprising a plurality of entities;
the initialization module is used for initializing the entities and the categories by utilizing a pre-constructed graph neural network model to obtain the embedded representation of each entity and the embedded representation of each category;
the category fusion module is used for obtaining the fusion embedded representation of the category by adopting an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories related to the category in the ontology view;
and the classification module is used for inputting the embedded representation of all the entities and the fused embedded representation of all the classes into a classification model and outputting the class corresponding to each entity through the classification model.
Based on the same inventive concept, the present disclosure also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.
Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described above.
From the above, it can be seen that the problem of knowledge graph entity classification can be solved more effectively and more fundamentally by the knowledge graph classification method based on meta-learning and the related equipment, and the traditional representation learning-based method, namely, the graph neural network, can be combined with the meta-learning method in the knowledge graph entity classification task under the small sample scene. In the method, the embedded representation of each entity and category is learned by a traditional representation learning method, and the parameters are learned and updated for multiple times by a meta-learning method, so that the accuracy of the entity classification of the knowledge map can be improved under the condition that the labeled entities of the knowledge map are sparse, the problem of long tail distribution of the entity classification task of the knowledge map is solved, and the generalization capability of a knowledge map classification model is improved.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the related art, the drawings needed to be used in the description of the embodiments or the related art will be briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of a knowledge graph according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of a flowchart of a method for classifying knowledge-graph entities based on meta-learning according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a knowledge-graph entity classification device based on meta-learning according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below with reference to the accompanying drawings in combination with specific embodiments.
It should be noted that technical terms or scientific terms used in the embodiments of the present application should have a general meaning as understood by those having ordinary skill in the art to which the present application belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the present application do not denote any order, quantity, or importance, but rather the terms are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
As described in the background, knowledge bases are often composed of ontology views containing abstract concepts and instance views composed of entities, fig. 1 shows a schematic structural diagram of an existing knowledge graph, which clearly shows the structure of the ontology views and the instance views in the knowledge graph, and in the ontology views, triples include head categories, meta relations, tail categories, such as Singer-is _ a-Artist; in the example view, the triples include head entities, relationships, and tail entities, such as Rose-Has _ aware-Nobel Prize. Between the two views, a large number of cross-view links are included for indicating the corresponding categories of the entities, for example, a pair of cross-view links is formed by David-Person.
Embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 2, the present application provides a method for classifying knowledge-graph entities based on meta-learning, which includes the following steps:
step S201, acquiring an open source knowledge graph data set, wherein the open source knowledge graph data set comprises an ontology view and an example view, the ontology view comprises a plurality of categories, and the example view comprises a plurality of entities.
Specifically, an open-source knowledge-graph dataset is downloaded, wherein the dataset comprises a large amount of triple information in an ontology view and an instance view. The currently common open source knowledge map data sets mainly comprise WordNet, cyc, DBpedia, YAGO, freebase, NELL, wikidata and the like, and the present embodiment adopts a subset YAGO26K-906 of YAGO (26K indicates that 26K subject words exist in the knowledge base, 906 indicates a total of 906 relations) and a subset DB111K-174 of DBpedia (111K indicates that 111K subject words exist in the knowledge base, 174 indicates a total of 174 relations) as data sets.
Step S202, initializing the entities and the categories by utilizing a pre-constructed graph neural network model, and obtaining the embedded representation of each entity and the embedded representation of each category.
Specifically, the knowledge graph data obtained in the above steps is marked with an undirected graph, so that the triplet information is converted into an adjacent matrix form, where the undirected graph is defined as G = (V, E, a), where V is a set of vertices, E is a set of edges, a is an adjacent matrix, and the size of the adjacent matrix is N × N.
The constructed graph neural network model is a Relational graph convolutional network model (R-GCN), the input of the graph neural network is the adjacency matrix and vertex attribute information of the graph, and the attribute information is optional, in this embodiment, a data set without attribute information is selected, and the initialization is to input the adjacency matrix obtained in the above step into the Relational graph convolutional network model, so as to obtain the embedded representation of each entity and the embedded representation of each category.
And step S203, obtaining a fused embedded representation of the category by adopting an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view.
Specifically, the Attention Mechanism (Attention Mechanism) is derived from the study of human vision, which is shown by selectively focusing on a part of all information while ignoring other visible information, and in the present embodiment, by giving different weights to different levels of category-embedded representations. And the element relation in the triple of the ontology view of the knowledge graph indicates the hierarchical relation among different categories, and the hierarchical information is extracted by using an attention mechanism.
Step S204, the embedded representations of all the entities and the fusion embedded representations of all the categories are input into a classification model, and the categories corresponding to all the entities are output through the classification model, wherein the classification model is pre-trained by adopting a meta-learning method.
Specifically, the classification model is used for classifying by comparing the similarity of the entity embedded representation and the category embedded representation.
The meta-learning is learning, classification tasks are divided into a training stage and a testing stage, and model parameters can be continuously optimized and updated through multiple times of training in the training stage; in the testing stage, a classification task is completed, a category in the ontology view corresponding to the category-missing entity in the instance view is found, and the quality of the model is further evaluated.
Based on steps S201 to S204, it can be seen that the classification model initializes the knowledge graph data set by using the graph neural network, optimizes category information by using the attention mechanism algorithm, and finally pre-trains the data by using the meta-learning method, so as to complete a knowledge graph entity classification task and dig out categories corresponding to entities with missing categories. The R-GCN is applied to the triple information of the instance view and the triple information of the ontology view, so that the internal structure information of the knowledge graph is utilized more comprehensively; the category embedding of different levels is fused by adopting an attention mechanism, so that the information utilization rate can be improved, and more comprehensive category embedding representation of information can be obtained; the meta-learning method is used for training the classification model, and through multiple times of small sample training, model parameters are updated in an iterative mode, so that the generalization and accuracy of the classification model can be improved, and the model can be quickly applied to a new task.
In some embodiments, the initializing the entities and the classes with the pre-constructed graph neural network model to obtain the embedded representation of each entity and the embedded representation of each class includes:
inputting the entity to the first graph neural network model, outputting an embedded representation of the entity via the first graph neural network model;
inputting the class to the second graph neural network model, outputting an embedded representation of the class via the second graph neural network model,
wherein the first graph neural network model and the second graph neural network model have different membrane layer structures.
Specifically, the first graph neural network model and the second graph neural network model are both relational graph convolution network models, namely R-GCN models. Wherein the first graph neural network model is used for initializing the entities in the instance view to obtain the embedded representation of each entity, the second graph neural network model is used for initializing the categories in the ontology view to obtain the embedded representation of each category, parameters of the first graph neural network model and the second graph neural network model are not shared, the first graph neural network model and the second graph neural network model both comprise a plurality of hidden layers, and the hidden layer information of the l-th layer is specifically:
wherein, the first and the second end of the pipe are connected with each other,
representing the neighbor node i under the relationship r,
a relationship weight matrix representing the relationship r,
a weight matrix representing the information of the fusion entity itself,
hidden layer information representing the l-th layer of a node,
denoted as the entity with the reference j, d (l) is the dimension represented by the layer. σ (-) denotes a non-linear activation function (e.g., reLU). c. C i，r Is a regularization constant, set to in this embodiment
Because the first graph neural network model and the second graph neural network model are respectively embedded with a plurality of entities and a plurality of categories, a plurality of relationships exist among the entities, and a plurality of element relationships exist among the categories, the first graph neural network model and the second graph neural network model are similar to a multi-relationship model, the parameter number and the relationship number of the multi-relationship model are increased quickly, and overfitting of a sparse matrix is easily caused in the calculation process
Is defined as:
wherein the content of the first and second substances,
as a basis for the transformation,
depending on the relation r, it is possible to,
and
are all coefficients that can be trained.
By the method, effective weight sharing among different relations can be realized, and the overfitting problem of sparse relation (rare relations) data can be reduced.
Based on this, an embedded representation of each entity is obtained by a first graph neural network model and a second graph neural network model, respectively
And embedded representation of each category
It should be noted that, because the ontology view has a small scale and the number of triples is small, the class information can be initialized well by using one layer of R-GCN, and meanwhile, the class representation tendency caused by too many layers is avoided. For the example view, because the scale is large and the triple information is rich, the entity information is initialized by using three layers of R-GCN, and the first graph neural network model in the embodiment comprises three hidden layers.
In some embodiments, said deriving a fused embedded representation of the category using an attention-based algorithm based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view comprises:
fusing the embedded representations of all sub-categories associated with the category by adopting an attention mechanism algorithm to obtain an initial fused embedded representation;
deriving the fused embedded representation based on the embedded representations of the classes, the initial fused embedded representation, and the weighting coefficients.
Specifically, the ontology view of the knowledge graph includes a large number of category triples with a head category structure, a meta-relationship structure and a tail category structure, the meta-relationship in the triples not only includes some attribute information, but also indicates a hierarchical relationship between different categories, and for example, the is _ a relationship and the subclass _ of relationship may indicate parent-child relationships of different categories. The traditional knowledge graph entity classification method does not explicitly model hierarchical information in the ontology view, so that the utilization rate of information is reduced, in order to explicitly model hierarchical relationships among different categories and provide rich data support for knowledge graph entity classification, the embodiment provides a hierarchical weak supervision information extraction method based on an attention mechanism, and the hierarchical weak supervision information extraction method is used for generating more comprehensive category embedded representation information.
The embedding of class y before optimization of the attention mechanism is represented as
In order to fuse category information of different levels, all connected subcategory information of the category needs to be fused:
where z represents all sub-categories of category y,
an embedded representation representing class z before being optimized by the attention mechanism,
a view of the body is represented, and,
representing the fusion of all connected sub-category embedded representations with category y, i.e., the initial fusion embedded representation, a (-) is the attention mechanism calculation mode.
The attention calculation method comprises the following steps:
wherein p and q are vectors.
The category embedding after fusing sub-category information by attention mechanism is represented as:
wherein the content of the first and second substances,
for the fused embedded representation, λ is a weight coefficient.
In some embodiments, the pre-training includes a meta-training phase and a meta-testing phase, and the classification model is pre-trained using a meta-learning method, including:
obtaining a cross-view link in an open-source knowledge-graph dataset, the cross-view link comprising a plurality of entity-category pairs;
dividing the cross-view link into a first category set and a second category set;
embedding all entities and all classes in the first class set and the second class set respectively through a graph convolution neural network model to obtain an embedded representation of each entity and an embedded representation of each class;
in the meta-training phase, training the classification model based on the first set of classes that are represented by the embedding;
in the meta-test phase, the classification model that has undergone the meta-training phase is trained based on the second set of classes that have undergone embedded representation.
Specifically, the first category set includes a large number of cross-view links, the second category set includes a small number of cross-view links, and the cross-view links include a plurality of entity-category pairs. Each category in the first set of categories corresponds to a large number of labeled entity-category pairs, and each category in the second set of categories corresponds to a small number of labeled entity-category pairs.
And in the meta-training stage, iterative training is carried out by using the cross-view link in the first class set to obtain the optimized initial parameters of the classification model.
And in the meta-test stage, testing the classification model by using the cross-view link in the second category set, and verifying the accuracy of the classification model.
In some embodiments, said training, in said meta-training phase, said classification model based on said first set of classes via embedded representations comprises:
transforming the embedded representations of the entities in the first set of classes of embedded representations into a class vector space using a non-linear activation function;
computationally determining an objective function based on the transformed embedded representation of the entity and the embedded representations of the classes in the first set of classes;
and dividing the first category set into a plurality of training sets, and performing iterative training on the classification model based on the objective function and the training sets.
Specifically, the first class set is divided into a plurality of subsets with the same structure, and the subsets are used as training sets in a meta-training phase, each training set comprises M classes, and each class embedded representation comprises K entities with labels.
For comparing similarity of entity embedded representations and class embedded representations, converting the embedded representations of entities in the first set of classes via embedded representations into a class vector space using the non-linear activation function:
wherein the content of the first and second substances,
representing an embedded representation of an entity in the class vector space, W ct ∈R d(t)×d(e) Representing the mapping of an entity-embedded representation from an entity vector space to a categoryMapping matrix used in the volume space, b ct Is an offset vector, σ (-) is a non-linear activation function, implemented in this embodiment using tanh (-).
In the class vector space, the classification model may be for a particular entity
Generating a distance-based distribution:
wherein the distance-based function is d: r M ×R M →[0，+∞)，
Represents an embedded representation of the class in vector space,
representing the embedded representation of the category k, k being the real category of the entity, k' being all categories under the task that need to be compared,
representing an embedded representation of the class k'.
The objective function of the model may be defined to maximize the probability distribution of the correct class for each entity, which is derived from the distance-based distribution described above, i.e.:
in order to optimize the parameters of the model as a whole in a gradient descent mode, a loss function is defined as a negative number of an objective function, namely:
wherein θ is all parameters related to the classification model, i.e. initialization parameters, and
and (6) associating.
In some embodiments, the iteratively training the classification model based on the objective function and the plurality of training sets comprises:
dividing each of the training sets in the plurality of training sets into a training group and a test group;
performing the following for each iteration of the iterative training:
calculating a loss value of the training set based on the target function, and calculating an updated parameter of the training set by adopting a gradient descent algorithm based on the loss value of the training set, the initialization parameter and the first learning rate;
calculating the loss value of the test group based on the target function and the updated parameters corresponding to all the training groups, and calculating by adopting a gradient descent algorithm based on the loss value of the test group, the initialization parameters and the second learning rate to obtain the updated initialization parameters.
Specifically, the ith training set is represented as
During iterative training, randomly screening partial training set in the first class set
And performing iterative training. Before iterative training, an iteration number is preset, and when the iteration number is reached, the training stage is ended.
The process of using the training set to perform iterative loop is referred to as inner loop, and the process of using the test set to perform iterative loop failure is referred to as outer loop.
The classification model represents the loss of the calculation of the objective function based on definition aiming at each entity embedding of the training set in each training set, and the loss is defined as
Calculating the updated parameters by adopting a gradient descent method, namely generating a specific group of parameter values for each training group:
wherein alpha is a first learning rate, theta is an initial parameter,
is a certain training set, f θ Representing the set of model parameters under the training set.
Then, the classification model calculates loss by using the test group samples according to the specific parameters sampled at random, and updates the overall parameters of the classification model:
where β is the second learning rate.
After the classification model is iteratively updated on a plurality of training sets, the optimal initialization parameters for all the training sets can be obtained, and when facing a new task, the model can learn effective information from a small number of labeled samples and quickly iterate to the scene of the new task.
In some embodiments, said testing, in said meta-testing stage, said classification model that has undergone a meta-training stage based on said embedded representation of said second set of classes comprises:
dividing the second category set into a plurality of test sets, and dividing each test set into a support group and a query group;
performing the following for the support groups in each of the test sets:
calculating a loss value of the support group based on the objective function, and calculating an updated parameter of the support group by adopting a gradient descent algorithm based on the loss value of the support group, the initialization parameter and the first learning rate;
testing the classification model based on the updated parameters of the support group and the query group.
Specifically, the second category set is divided into a plurality of subsets with the same structure, and each subset is used as a test set in the meta-test stage, wherein each test set comprises M categories, each category embedded representation comprises K entities with marks, and each test set is divided into a support group and an inquiry group.
As with the training portion above, the classification model embeds for each entity of the support group in each test set a representation that calculates a loss based on a defined objective function, the loss being defined as
Calculating the updated parameters by adopting a gradient descent method to obtain a specific initialization parameter set of the test set:
where alpha is the first learning rate, theta is the initialization parameter,
is a certain test set.
Applying classification models to query groups of the same test set, with a fine-tuned initialization parameter set θ' i Generating a category for each entity in the set of queries to test the effectiveness of the classification model.
The meta-learning based knowledge-graph entity classification method of the present application is described below by comparing with the classical classification method in the related art:
before the comparison and the description, the main evaluation indexes of the knowledge graph entity classification are described.
The main evaluation indexes of the knowledge graph entity classification comprise the following three indexes:
1. average Reciprocal Rank MRR (Mean Reciprocal Rank), that is, the average Reciprocal Rank on all positive categories is calculated, if the first result is the correct category, the score is 1, the second result is the correct category, the score is 0.5, … …, and if the positive category is ranked N, the matching score is 1/N. If the correct category does not appear in the result, the score is 0. After the calculation is completed, all the scores are added to obtain an average value, and the calculation formula is as follows:
wherein Q is the total number of tasks, rank i Indicating the ranking of the first correct answer for the ith task.
In a knowledge graph entity classification task, firstly, training data is used for obtaining each entity embedded representation and category embedded representation, in a test set, the distance between an entity and representations of different categories is calculated, descending ranking is carried out based on the distance, and the MRR value of each model is obtained, wherein the higher the MRR value is, the stronger the potential generalization capability of the model is.
Hit @1, the Accuracy Accuracy, acc for short, indicates the first proportion of the correct category rank calculated in all tasks, and the calculation formula is:
wherein Q is the total number of tasks, top i And the positive category of the ith task is shown to be ranked first, if so, the value is 1, and if not, the value is 0.
Hit @3 represents the proportion of the first three of the positive category ranking, and the calculation formula is as follows:
wherein Q is the total number of tasks, top 3i Whether the positive category of the ith task is ranked first three or notIf so, the value is 1, otherwise the value is 0. The larger the values of Hit @1 and Hit @3 are, the higher the accuracy of the model is.
The classification model of the invention is named as Meta Knowledge Graph Entity Typing, abbreviated as MKGET. Comparing the invention with classical knowledge graph entity classification methods ConnectE, R-GCN, transE, distMult, hoIE, joIE, MTransE and the like, the experimental setup is as follows: the embedding dimension of the entity is set to 200, the embedding dimension of the category is set to 100, each training set and test set is set to 10 types, and each type has 3 marked samples. For the two-layer loop of meta learning, the inner-layer loop learning rate α is set to 0.001, and the outer-layer loop learning rate β is set to 0.01.
TABLE 1 entity Classification test results Table
Table 1 shows that on two data sets YAGO26K-906 and DB111K-174, the values of the MKGET model provided by the application on three indexes of MRR, acc and Hit @3 are all larger than that of the existing classification model, and the generalization and accuracy of the classification model are better than those of the existing model.
This is because: (1) The MKGET model considers the triple structures in the two views, and utilizes the internal structural information more comprehensively. (2) The MKGET model employs an attention mechanism to incorporate hierarchical information in an ontology view into an embedded representation of a category. (3) The MKGET model adopts a meta-learning training mechanism, and can obtain a good classification effect under the scene of sparse labeled samples.
It should be noted that the method of the embodiment of the present application may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment, and the multiple devices interact with each other to complete the method.
It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, corresponding to the method of any embodiment, the application also provides a knowledge graph entity classification device.
Referring to fig. 3, the apparatus for knowledge-graph entity classification includes:
an obtaining module 301, configured to obtain an open-source knowledge-graph dataset, where the open-source knowledge-graph dataset includes an ontology view and an instance view, the ontology view includes a plurality of categories, and the instance view includes a plurality of entities;
an initialization module 302, which initializes the entities and the classes by using a pre-constructed graph neural network model to obtain an embedded representation of each entity and an embedded representation of each class;
a category fusion module 303, configured to obtain a fusion embedded representation of the category by using an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view;
the classification module 304 inputs the embedded representations of all the entities and the fused embedded representations of all the categories into a classification model, and outputs the category corresponding to each of the entities via the classification model.
For convenience of description, the above devices are described as being divided into various modules by functions, which are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations as the present application.
The apparatus of the foregoing embodiment is used to implement the corresponding method for classifying knowledge-graph entities based on meta-learning in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the meta-learning-based knowledge graph entity classification method according to any of the above embodiments is implemented.
Fig. 4 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called by the processor 1010 for execution.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, bluetooth and the like).
The bus 1050 includes a path to transfer information between various components of the device, such as the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040.
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the foregoing embodiment is used to implement the corresponding method for classifying knowledge-graph entities based on meta-learning in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to any of the above-described embodiment methods, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the meta-learning based knowledge-graph entity classification method according to any of the above embodiments.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for classifying knowledge-graph entities based on meta-learning according to any of the above embodiments, and have the beneficial effects of corresponding method embodiments, and will not be described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the context of the present application, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the application. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the application, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the application are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that the embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures, such as Dynamic RAM (DRAM), may use the discussed embodiments.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present application are intended to be included within the scope of the present application.
Claims (10)
1. A knowledge graph entity classification method based on meta-learning is characterized by comprising the following steps:
obtaining an open source knowledge graph dataset comprising an ontology view comprising a plurality of categories and an instance view comprising a plurality of entities;
initializing the entities and the classes by utilizing a pre-constructed graph neural network model to obtain embedded representation of each entity and embedded representation of each class;
based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view, obtaining a fused embedded representation of the category by adopting an attention mechanism algorithm;
and inputting the embedded representation of all the entities and the fused embedded representation of all the categories into a classification model, and outputting the category corresponding to each entity through the classification model, wherein the classification model is pre-trained by adopting a meta-learning method.
2. The method of claim 1, wherein the graph neural network model comprises a first graph neural network model and a second graph neural network model, and wherein initializing the entities and the classes using a pre-constructed graph neural network model to obtain the embedded representation of each entity and the embedded representation of each class comprises:
inputting the entity to the first graph neural network model, outputting an embedded representation of the entity via the first graph neural network model;
inputting the class to the second graph neural network model, outputting an embedded representation of the class via the second graph neural network model,
wherein the first graph neural network model and the second graph neural network model have different membrane layer structures.
3. The method of claim 1, wherein obtaining the fused embedded representation of the category using an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories associated with the category in the ontology view comprises:
fusing the embedded representations of all sub-categories related to the category by adopting an attention mechanism algorithm to obtain an initial fusion embedded representation;
deriving the fused embedded representation based on the embedded representations of the classes, the initial fused embedded representation, and the weighting coefficients.
4. The method of claim 1, wherein the pre-training comprises a meta-training phase and a meta-testing phase, and wherein the classification model is pre-trained using a meta-learning method comprising:
obtaining a cross-view link in an open source knowledge-graph dataset, the cross-view link comprising a plurality of entity-category pairs;
dividing the cross-view link into a first category set and a second category set;
embedding all entities and all classes in the first class set and the second class set respectively through a graph convolution neural network model to obtain an embedded representation of each entity and an embedded representation of each class;
in the meta-training phase, training the classification model based on the first set of classes that are represented by the embedding;
in the meta-test phase, the classification model that has undergone the meta-training phase is trained based on the second set of classes that have undergone embedded representation.
5. The method of claim 4, wherein training the classification model based on the first set of classes via embedded representation in the meta-training phase comprises:
transforming the embedded representations of the entities in the first set of classes of embedded representations into a class vector space using a non-linear activation function;
computationally determining an objective function based on the transformed embedded representation of the entity and the embedded representations of the classes in the first set of classes;
and dividing the first category set into a plurality of training sets, and performing iterative training on the classification model based on the target function and the training sets.
6. The method of claim 5, wherein iteratively training the classification model based on the objective function and the plurality of training sets comprises:
dividing each of the training sets in the plurality of training sets into a training group and a test group;
performing the following for each iteration of the iterative training:
calculating a loss value of the training set based on the target function, and calculating an updated parameter of the training set by adopting a gradient descent algorithm based on the loss value of the training set, the initialization parameter and the first learning rate;
calculating the loss value of the test group based on the target function and the updated parameters corresponding to all the training groups, and calculating by adopting a gradient descent algorithm based on the loss value of the test group, the initialization parameters and the second learning rate to obtain the updated initialization parameters.
7. The method of claim 6, wherein in the meta-testing stage, testing the classification model in the meta-training stage based on the embedded representation of the second class set comprises:
dividing the second category set into a plurality of test sets, and dividing each test set into a support group and a query group;
performing the following for the support groups in each of the test sets:
calculating a loss value of the support group based on the objective function, and calculating an updated parameter of the support group by adopting a gradient descent algorithm based on the loss value of the support group, the initialization parameter and the first learning rate;
testing the classification model based on the updated parameters of the support group and the query group.
8. An apparatus, comprising:
an acquisition module that acquires an open-source knowledge-graph dataset, the open-source knowledge-graph dataset including an ontology view and an instance view, the ontology view including a plurality of categories, the instance view including a plurality of entities;
the initialization module is used for initializing the entities and the classes by utilizing a pre-constructed graph neural network model to obtain the embedded representation of each entity and the embedded representation of each class;
the category fusion module is used for obtaining the fusion embedded representation of the category by adopting an attention mechanism algorithm based on the embedded representation of the category and the embedded representations of all sub-categories related to the category in the ontology view;
and the classification module is used for inputting the embedded representation of all the entities and the fused embedded representation of all the classes into a classification model and outputting the class corresponding to each entity through the classification model.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the program.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210610864.XA CN115169433A (en) | 2022-05-30 | 2022-05-30 | Knowledge graph classification method based on meta-learning and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210610864.XA CN115169433A (en) | 2022-05-30 | 2022-05-30 | Knowledge graph classification method based on meta-learning and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115169433A true CN115169433A (en) | 2022-10-11 |
Family
ID=83483022
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210610864.XA Pending CN115169433A (en) | 2022-05-30 | 2022-05-30 | Knowledge graph classification method based on meta-learning and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115169433A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117040926A (en) * | 2023-10-08 | 2023-11-10 | 北京网藤科技有限公司 | Industrial control network security feature analysis method and system applying knowledge graph |
-
2022
- 2022-05-30 CN CN202210610864.XA patent/CN115169433A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117040926A (en) * | 2023-10-08 | 2023-11-10 | 北京网藤科技有限公司 | Industrial control network security feature analysis method and system applying knowledge graph |
| CN117040926B (en) * | 2023-10-08 | 2024-01-26 | 北京网藤科技有限公司 | Industrial control network security feature analysis method and system applying knowledge graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106095893B (en) | A kind of cross-media retrieval method | |
| WO2020094060A1 (en) | Recommendation method and apparatus | |
| CN110532417B (en) | Image retrieval method and device based on depth hash and terminal equipment | |
| Jing et al. | Predicting image memorability through adaptive transfer learning from external sources | |
| CN110046698A (en) | Heterogeneous figure neural network generation method, device, electronic equipment and storage medium | |
| US20220222918A1 (en) | Image retrieval method and apparatus, storage medium, and device | |
| US11023473B2 (en) | Recurrent binary embedding for information retrieval | |
| CN111563192B (en) | Entity alignment method, device, electronic equipment and storage medium | |
| CN112667877A (en) | Scenic spot recommendation method and equipment based on tourist knowledge map | |
| US20190286978A1 (en) | Using natural language processing and deep learning for mapping any schema data to a hierarchical standard data model (xdm) | |
| CN115238855A (en) | Completion method of time sequence knowledge graph based on graph neural network and related equipment | |
| CN112395487A (en) | Information recommendation method and device, computer-readable storage medium and electronic equipment | |
| KR20220047228A (en) | Method and apparatus for generating image classification model, electronic device, storage medium, computer program, roadside device and cloud control platform | |
| Chen et al. | RRGCCAN: Re-ranking via graph convolution channel attention network for person re-identification | |
| US20220114644A1 (en) | Recommendation system with sparse feature encoding | |
| CN111090765A (en) | Social image retrieval method and system based on missing multi-modal hash | |
| CN115169433A (en) | Knowledge graph classification method based on meta-learning and related equipment | |
| Yusuf et al. | Evaluation of graph convolutional networks performance for visual question answering on reasoning datasets | |
| JP2022541832A (en) | Method and apparatus for retrieving images | |
| CN113139540B (en) | Backboard detection method and equipment | |
| US11836187B2 (en) | Generating occurrence contexts for objects in digital content collections | |
| CN115952317A (en) | Video processing method, device, equipment, medium and program product | |
| CN114238746A (en) | Cross-modal retrieval method, device, equipment and storage medium | |
| CN110826726B (en) | Target processing method, target processing device, target processing apparatus, and medium | |
| CN114493674A (en) | Advertisement click rate prediction model and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |