CN113111134A - Self-coding and attention mechanism-based heterogeneous graph node feature embedding method - Google Patents
Self-coding and attention mechanism-based heterogeneous graph node feature embedding method Download PDFInfo
- Publication number
- CN113111134A CN113111134A CN202110428607.XA CN202110428607A CN113111134A CN 113111134 A CN113111134 A CN 113111134A CN 202110428607 A CN202110428607 A CN 202110428607A CN 113111134 A CN113111134 A CN 113111134A
- Authority
- CN
- China
- Prior art keywords
- node
- nodes
- slave
- main
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/288—Entity relationship models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A heterogeneous graph node feature embedding method based on self-coding and attention mechanism is established on a heterogeneous graph constructed by a master node, a slave node and a corresponding relation. The master node is associated with the master node, and the master node comprises slave nodes. And aggregating the characteristics of the slave nodes through the inclusion relationship and coding the characteristics to the master node. And then, fusing neighbor characteristics around the main node through the incidence relation to obtain the main node embedding expression. And finally, judging whether the master node is embedded and represented by the slave node before coding or not by calculating the similarity between the slave node before coding and the slave node after decoding. Compared with other methods, the method has the advantages that node embedding is realized in the abnormal composition graph, firstly, from the angle of graph data structure, the processing range is wider, and the same composition graph is converted into the abnormal composition graph; secondly, through the operations of aggregation-fusion-reverse aggregation, the final result is more representable by transmitting the characteristic information of different types of nodes.
Description
Technical Field
The invention relates to the technical field of heterogeneous graph node embedding, in particular to a heterogeneous graph node feature embedding method based on self-coding and attention mechanism.
Background
The graph shows the most direct data representation among entities in real application. But the scale is large, the dimension is high, the application is difficult, and the development of the graph embedding technology and the graph neural network solves the problem. While the related art map embedding technique is sufficient to compress high weft data to low weft, it still faces the following problems:
1) mostly, the embedding representation is carried out on all nodes in the complete graph or the graph, and the embedding representation cannot be carried out on partial nodes in the graph
2) Most of the methods are embedded and expressed for the same composition, and the expression methods related to different compositions are relatively few.
Disclosure of Invention
In order to overcome the defects of the technology, the invention provides a heterogeneous graph node feature embedding method based on self-coding and attention mechanism, which has wide processing range and more representable final result.
The technical scheme adopted by the invention for overcoming the technical problems is as follows:
a heterogeneous graph node feature embedding method based on self-coding and attention mechanism comprises the following steps:
a) constructing an abnormal composition picture, wherein the abnormal composition picture is composed of a plurality of main nodes and a plurality of slave nodes, the main nodes contain one type, the slave nodes contain a plurality of sub-types, the main nodes are mutually associated to form an association relationship, and the main nodes and the slave nodes are associated to form an inclusion relationship;
b) aggregating slave node characteristics contained in the master node;
c) traversing the relation contained in the current master node, and taking the node characteristics of all slave nodes of the contained relation formed by one master node;
d) repeating the step b) and the step c) until the relations included by all the main nodes in the abnormal graph are traversed, and storing the original characteristic of the main nodes as featRmainUpdating the characteristics of all the main nodes to obtain a primary representation of the main nodes;
e) by means of the incidence relation among the main nodes, the attention neural network GAT is used as a fusion device, an SELU activation function is adopted, and the attention mechanism is combined to fuse the characteristic information of the main node and the first-order neighbor nodes of the main node, so that the characteristics of all the main nodes are updated, and main node embedding representation is obtained;
f) reversely aggregating the main nodes to obtain the characteristics of the slave nodes;
g) calculating the similarity between all the slave node characteristics in the step f) and all the slave node characteristics in the step b);
h) repeating the steps b) to g) for N times or more, and taking the embedded representation in the step e) as the final embedded representation of the master node in the abnormal picture.
Further, in step b), the formula is used
Calculating to obtain the corresponding characteristic of the e-th inclusion relation of the ith host node
Wherein SELU (-) is the SELU activation function, GCN (-) is the GCN graph convolution process,
as the jth slave node feature, N (main)i) The number of slave nodes corresponds to the ith master node.
Further, in step c), the formula is used
Calculating the node characteristics of all the slave nodes of the inclusion relationship formed by the ith master node
In the formula
And for the number of edge types contained between all slave nodes of the containing relationship formed by the ith master node and the ith master node, mean (-) is calculated by taking the average value, and concat (-) is splicing operation.
Further, all the main nodes eat through assignment operation in the step d)mainUpdating to obtain a preliminary representation.
Further, in the step e), all the main nodes are feat through assignment operationmainAnd updating to obtain the embedded representation of the main node.
Further, step f) is performed by the formula
Calculating the corresponding characteristics of the ith slave node after obtaining the reverse set
The node characteristics of all slave nodes constituting the containing relation for the j-th master node,
the original characteristics of the j master nodes.
Further, in step g), the formula Loss ═ SmoothL1 (feat) is usedNsub,featsub) Calculating all slave node characteristics feat after obtaining reverse setNsubAnd all slave node primitive features featsubThe similarity difference Loss, SmoothL1(.) of (A) is a SmoothL1 Loss function.
Preferably, N is 100.
The invention has the beneficial effects that: the method is established on an abnormal graph constructed by the master node, the slave node and the corresponding relation. The master node is associated with the master node, and the master node comprises slave nodes. And aggregating the characteristics of the slave nodes through the inclusion relationship and coding the characteristics to the master node. And then, fusing neighbor characteristics around the main node through the incidence relation to obtain the main node embedding expression. And finally, judging whether the master node is embedded and represented by the slave node before coding or not by calculating the similarity between the slave node before coding and the slave node after decoding. Compared with other methods, the method has the advantages that node embedding is realized in the abnormal composition graph, firstly, from the angle of graph data structure, the processing range is wider, and the same composition graph is converted into the abnormal composition graph; secondly, through the operations of aggregation-fusion-reverse aggregation, the final result is more representable by transmitting the characteristic information of different types of nodes.
Drawings
FIG. 1 is a diagram illustrating the relationship between master and slave nodes of a heterogeneous graph
FIG. 2 is a flow chart of the method of the present invention.
Detailed Description
The invention will be further explained with reference to fig. 1 and 2.
A heterogeneous graph node feature embedding method based on self-coding and attention mechanism comprises the following steps:
a) as shown in fig. 1, an abnormal graph is constructed, the abnormal graph is composed of a plurality of main nodes and a plurality of slave nodes, the main nodes are of one type, the slave nodes contain a plurality of sub-types, and the abnormal graph is a simple graph without isolated points. The main nodes are associated with each other to form an association relation, and the main nodes and the slave nodes are associated with each other to form an inclusion relation. That is, the master node includes slave nodes, and the slave nodes have no correlation with each other.
b) The master node of the aggregation contains slave node characteristics.
c) And traversing the relation contained by the current master node, and taking the node characteristics of all slave nodes of the contained relation formed by one master node.
d) Repeating the step b) and the step c) until the relations included by all the main nodes in the abnormal graph are traversed, and storing the original characteristic of the main nodes as featRmainAnd updating all the main node characteristics to obtain a primary representation of the main node characteristics.
e) By means of the incidence relation among the main nodes, the attention neural network GAT is used as a fusion device, an SELU activation function is adopted, and the attention mechanism is combined to fuse the characteristic information of the main node and the first-order neighbor nodes of the main node, so that the characteristics of all the main nodes are updated, and the embedded expression of the main nodes is obtained.
f) And reversely aggregating the master node to obtain the slave node characteristics.
g) Calculating the similarity of all the slave node characteristics in the step f) and all the slave node characteristics in the step b).
h) Repeating the steps b) to g) for N times or more, and taking the embedded representation in the step e) as the final embedded representation of the master node in the abnormal picture.
The method is established on an abnormal graph constructed by the master node, the slave node and the corresponding relation. The master node is associated with the master node, and the master node comprises slave nodes. And aggregating the characteristics of the slave nodes through the inclusion relationship and coding the characteristics to the master node. And then, fusing neighbor characteristics around the main node through the incidence relation to obtain the main node embedding expression. And finally, judging whether the master node is embedded and represented by the slave node before coding or not by calculating the similarity between the slave node before coding and the slave node after decoding. Compared with other methods, the method has the advantages that node embedding is realized in the abnormal composition graph, firstly, from the angle of graph data structure, the processing range is wider, and the same composition graph is converted into the abnormal composition graph; secondly, through the operations of aggregation-fusion-reverse aggregation, the final result is more representable by transmitting the characteristic information of different types of nodes.
Example 1:
in step b) by the formula
Calculating to obtain the corresponding characteristic of the e-th inclusion relation of the ith host node
Wherein SELU (-) is the SELU activation function, GCN (-) is the GCN graph convolution process,
as the jth slave node feature, N (main)i) The number of slave nodes corresponds to the ith master node.
Example 2:
in step c) by the formula
Calculating the node characteristics of all the slave nodes of the inclusion relationship formed by the ith master node
In the formula
And for the number of edge types contained between all slave nodes of the containing relationship formed by the ith master node and the ith master node, mean (-) is calculated by taking the average value, and concat (-) is splicing operation.
Example 3:
in the step d), all the main nodes are feat through assignment operationmainUpdating to obtain a preliminary representation.
Example 4:
in the step e), all the main nodes are feat through assignment operationmainAnd updating to obtain the embedded representation of the main node.
Example 5:
in step f) by the formula
Calculating the corresponding characteristics of the ith slave node after obtaining the reverse set
The node characteristics of all slave nodes constituting the containing relation for the j-th master node,
the original characteristics of the j master nodes.
Example 6:
step g) by the formula Loss SmoothL1 (feat)Nsub,featsub) Calculating all slave node characteristics feat after obtaining reverse setNsubAnd all slave node primitive features featsubThe similarity difference Loss, SmoothL1(.) of (A) is a SmoothL1 Loss function.
Example 7:
and N takes the value of 100.
Claims (8)
1. A heterogeneous graph node feature embedding method based on self-coding and attention mechanism is characterized by comprising the following steps:
a) constructing an abnormal composition picture, wherein the abnormal composition picture is composed of a plurality of main nodes and a plurality of slave nodes, the main nodes contain one type, the slave nodes contain a plurality of sub-types, the main nodes are mutually associated to form an association relationship, and the main nodes and the slave nodes are associated to form an inclusion relationship;
b) aggregating slave node characteristics contained in the master node;
c) traversing the relation contained in the current master node, and taking the node characteristics of all slave nodes of the contained relation formed by one master node;
d) repeating the step b) and the step c) until the relations included by all the main nodes in the abnormal graph are traversed, and storing the original characteristic of the main nodes as featRmainUpdating the characteristics of all the main nodes to obtain a primary representation of the main nodes;
e) by means of the incidence relation among the main nodes, the attention neural network GAT is used as a fusion device, an SELU activation function is adopted, and the attention mechanism is combined to fuse the characteristic information of the main node and the first-order neighbor nodes of the main node, so that the characteristics of all the main nodes are updated, and main node embedding representation is obtained;
f) reversely aggregating the main nodes to obtain the characteristics of the slave nodes;
g) calculating the similarity between all the slave node characteristics in the step f) and all the slave node characteristics in the step b);
h) repeating the steps b) to g) for N times or more, and taking the embedded representation in the step e) as the final embedded representation of the master node in the abnormal picture.
2. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 1, wherein: in step b) by the formula
Calculating to obtain the corresponding characteristic of the e-th inclusion relation of the ith host node
Wherein SELU (-) is the SELU activation function, GCN (-) is the GCN graph convolution process,
as the jth slave node feature, N (main)i) The number of slave nodes corresponds to the ith master node.
3. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 2, wherein: in step c) by the formula
Calculating the node characteristics of all the slave nodes of the inclusion relationship formed by the ith master node
In the formula
And for the number of edge types contained between all slave nodes of the containing relationship formed by the ith master node and the ith master node, mean (-) is calculated by taking the average value, and concat (-) is splicing operation.
4. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 3, wherein: in the step d), all the main nodes are feat through assignment operationmainUpdating to obtain a preliminary representation.
5. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 3, wherein: in the step e), all the main nodes are feat through assignment operationmainAnd updating to obtain the embedded representation of the main node.
6. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 3, wherein: in step f) by the formula
Is calculated toFeature corresponding to ith slave node after reverse set
The node characteristics of all slave nodes constituting the containing relation for the j-th master node,
the original characteristics of the j master nodes.
7. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 6, wherein: step g) by the formula Loss SmoothL1 (feat)Nsub,featsub) Calculating all slave node characteristics feat after obtaining reverse setNsubAnd all slave node primitive features featsubThe similarity difference Loss, SmoothL1(.) of (A) is a SmoothL1 Loss function.
8. The self-coding and attention mechanism-based heterogeneous graph node feature embedding method according to claim 1, wherein: and N takes the value of 100.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110428607.XA CN113111134A (en) | 2021-04-21 | 2021-04-21 | Self-coding and attention mechanism-based heterogeneous graph node feature embedding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110428607.XA CN113111134A (en) | 2021-04-21 | 2021-04-21 | Self-coding and attention mechanism-based heterogeneous graph node feature embedding method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113111134A true CN113111134A (en) | 2021-07-13 |
Family
ID=76719228
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110428607.XA Pending CN113111134A (en) | 2021-04-21 | 2021-04-21 | Self-coding and attention mechanism-based heterogeneous graph node feature embedding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111134A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115798722A (en) * | 2023-02-02 | 2023-03-14 | 神州医疗科技股份有限公司 | Immune drug population high-low risk screening method and system based on knowledge graph |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140258196A1 (en) * | 2013-03-07 | 2014-09-11 | International Business Machines Corporation | System and method for using graph transduction techniques to make relational classifications on a single connected network |
| US20190258721A1 (en) * | 2018-02-19 | 2019-08-22 | Microsoft Technology Licensing, Llc | Standardized entity representation learning for smart suggestions |
| US20190311811A1 (en) * | 2018-04-07 | 2019-10-10 | Tata Consultancy Services Limited | Graph convolution based gene prioritization on heterogeneous networks |
| CN110598061A (en) * | 2019-09-20 | 2019-12-20 | 东北大学 | Multi-element graph fused heterogeneous information network embedding method |
| CN111241311A (en) * | 2020-01-09 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Media information recommendation method and device, electronic equipment and storage medium |
| CN111310068A (en) * | 2020-03-11 | 2020-06-19 | 广东工业大学 | Social network node classification method based on dynamic graph |
| CN111368552A (en) * | 2020-02-26 | 2020-07-03 | 北京市公安局 | Network user group division method and device for specific field |
| CN111597358A (en) * | 2020-07-22 | 2020-08-28 | 中国人民解放军国防科技大学 | Knowledge graph reasoning method and device based on relational attention and computer equipment |
| CN112132188A (en) * | 2020-08-31 | 2020-12-25 | 浙江工业大学 | E-commerce user classification method based on network attributes |
| CN112287123A (en) * | 2020-11-19 | 2021-01-29 | 国网湖南省电力有限公司 | Entity alignment method and device based on edge type attention mechanism |
| CN112434720A (en) * | 2020-10-22 | 2021-03-02 | 暨南大学 | Chinese short text classification method based on graph attention network |
-
2021
- 2021-04-21 CN CN202110428607.XA patent/CN113111134A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140258196A1 (en) * | 2013-03-07 | 2014-09-11 | International Business Machines Corporation | System and method for using graph transduction techniques to make relational classifications on a single connected network |
| US20190258721A1 (en) * | 2018-02-19 | 2019-08-22 | Microsoft Technology Licensing, Llc | Standardized entity representation learning for smart suggestions |
| US20190311811A1 (en) * | 2018-04-07 | 2019-10-10 | Tata Consultancy Services Limited | Graph convolution based gene prioritization on heterogeneous networks |
| CN110598061A (en) * | 2019-09-20 | 2019-12-20 | 东北大学 | Multi-element graph fused heterogeneous information network embedding method |
| CN111241311A (en) * | 2020-01-09 | 2020-06-05 | 腾讯科技（深圳）有限公司 | Media information recommendation method and device, electronic equipment and storage medium |
| CN111368552A (en) * | 2020-02-26 | 2020-07-03 | 北京市公安局 | Network user group division method and device for specific field |
| CN111310068A (en) * | 2020-03-11 | 2020-06-19 | 广东工业大学 | Social network node classification method based on dynamic graph |
| CN111597358A (en) * | 2020-07-22 | 2020-08-28 | 中国人民解放军国防科技大学 | Knowledge graph reasoning method and device based on relational attention and computer equipment |
| CN112132188A (en) * | 2020-08-31 | 2020-12-25 | 浙江工业大学 | E-commerce user classification method based on network attributes |
| CN112434720A (en) * | 2020-10-22 | 2021-03-02 | 暨南大学 | Chinese short text classification method based on graph attention network |
| CN112287123A (en) * | 2020-11-19 | 2021-01-29 | 国网湖南省电力有限公司 | Entity alignment method and device based on edge type attention mechanism |
Non-Patent Citations (5)
| Title |
|---|
| PING XUAN ET AL.: "Graph Convolutional Autoencoder and Fully-Connected Autoencoder with Attention Mechanism Based Method for Predicting Drug-Disease Associations", 《IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS》 * |
| ZIQIANG WENG ET AL.: "Adversarial Attention-Based Variational Graph Autoencoder", 《IEEE ACCESS》 * |
| 刘忠雨 等: "《深入浅出图神经网络GNN原理解析》", 30 April 2020 * |
| 李钊 等: "信息物理系统安全威胁与措施", 《清华大学学报(自然科学版)》 * |
| 潘承瑞 等: "融合知识图谱的双线性图注意力网络推荐算法", 《计算机工程与应用》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115798722A (en) * | 2023-02-02 | 2023-03-14 | 神州医疗科技股份有限公司 | Immune drug population high-low risk screening method and system based on knowledge graph |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532859B (en) | Remote sensing image target detection method based on deep evolution pruning convolution net | |
| CN107609587B (en) | Multi-class multi-view data generation method for generating countermeasure network based on deep convolution | |
| CN108920720B (en) | Large-scale image retrieval method based on depth hash and GPU acceleration | |
| CN112714032B (en) | Wireless network protocol knowledge graph construction analysis method, system, equipment and medium | |
| CN110515931B (en) | Capacitive type equipment defect prediction method based on random forest algorithm | |
| CN110598061A (en) | Multi-element graph fused heterogeneous information network embedding method | |
| CN113191530B (en) | Block link point reliability prediction method and system with privacy protection function | |
| CN112541926B (en) | Ambiguous pixel optimization segmentation method based on improved FCN and Densenet | |
| CN114332104B (en) | Power grid power transmission scene RGB point cloud semantic segmentation multi-stage model joint optimization method | |
| CN113240683A (en) | Attention mechanism-based lightweight semantic segmentation model construction method | |
| CN113628059A (en) | Associated user identification method and device based on multilayer graph attention network | |
| CN114742985A (en) | Hyperspectral feature extraction method and device and storage medium | |
| CN113111134A (en) | Self-coding and attention mechanism-based heterogeneous graph node feature embedding method | |
| CN117237559A (en) | Digital twin city-oriented three-dimensional model data intelligent analysis method and system | |
| CN113228059A (en) | Cross-network-oriented representation learning algorithm | |
| CN112396137A (en) | Point cloud semantic segmentation method fusing context semantics | |
| CN113763364A (en) | Image defect detection method based on convolutional neural network | |
| CN111539517A (en) | Graph convolution neural network generation method based on graph structure matrix characteristic vector | |
| CN115455302A (en) | Knowledge graph recommendation method based on optimized graph attention network | |
| CN114611668A (en) | Vector representation learning method and system based on heterogeneous information network random walk | |
| CN115907829A (en) | User account analysis method and system based on artificial intelligence | |
| CN115130663A (en) | Heterogeneous network attribute completion method based on graph neural network and attention mechanism | |
| CN114049431A (en) | Efficient network transmission method for cultural relic three-dimensional model remote rendering data | |
| CN113886547A (en) | Client real-time conversation switching method and device based on artificial intelligence and electronic equipment | |
| KR102305981B1 (en) | Method for Training to Compress Neural Network and Method for Using Compressed Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210713 |