CN114254738A

CN114254738A - Double-layer evolvable dynamic graph convolution neural network model construction method and application

Info

Publication number: CN114254738A
Application number: CN202111543402.2A
Authority: CN
Inventors: 刘琰; 钟凤喆; 魏亮; 杨春芳; 贾萌萌
Original assignee: Information Engineering University of PLA Strategic Support Force
Current assignee: Information Engineering University of PLA Strategic Support Force
Priority date: 2021-12-16
Filing date: 2021-12-16
Publication date: 2022-03-29

Abstract

The invention discloses a method for constructing a double-layer evolved dynamic graph convolution neural network model and application thereof, wherein the method comprises the following steps: based on a graph convolution neural network GCN, a recurrent neural network RNN is utilized to perform double-layer evolution on network parameters and node embedding of the GCN respectively, the change of the structural characteristics of graph data is captured, meanwhile, the stability of the node embedding on time is ensured, a loss function is constructed by utilizing a node embedding evolution result and a result generated by the GCN, the GCN network parameters are corrected, an unsupervised dynamic graph representation learning frame is formed, and the construction of a double-layer evolved dynamic graph convolution neural network model DEDGCN is completed. The model can capture the stability characteristics while learning the dynamic characteristics of the nodes, and a loss function is constructed by utilizing the stability characteristics of the nodes and the structural characteristics of the graph, so that the model can be applied to unsupervised tasks and is suitable for application tasks of graph data in various scenes.

Description

Double-layer evolvable dynamic graph convolution neural network model construction method and application

Technical Field

The invention belongs to the technical field of graph data processing, and particularly relates to a method for constructing a double-layer evolved dynamic graph convolution neural network model and application thereof.

Background

In real life, many complex systems are composed of graph data, including social networks, financial fields, biomolecules, e-commerce platforms, knowledge maps and the like, which can be converted into graph data composed of nodes and node interaction forms. The study of the graph data by the scholars also pays more and more attention. However, due to the characteristics of various forms and complex connection relationships of graph data, the graph data is more difficult to represent than other types of data, and how to present elements such as nodes, edges, attributes and the like in the graph in a stable form becomes an important link for graph data research. Particularly, with the successful application of deep learning technology in many fields, the expression form of converting graph data into vectors is the most popular graph representation learning mode at present, and the mature technology mainly comprises random walk-based Deepwalk, Node2vec, matrix decomposition-based GraRep, graph structure characteristic-based struc2vec, DyREP, neural network-based SDNE and the like. Particularly, the rise of Graph Neural Network (GNN) brings a new direction for the research of Graph representation learning technology, and makes it advance to a new development stage.

However, in real life, the map data is not constant, but evolves over time. Such as an exchange of friends in a friendship network, an update of devices in a computer topology network, an increase and decrease of users and goods in a user goods network, and so forth. The graph data is often provided with a time attribute in the interaction process, for example, in an e-commerce platform, a transaction record of a user purchasing a commodity is provided with the time attribute, a call between users in a telephone network is provided with the time attribute, and a mail in a mail network is sent with the time attribute. The existing popular deep learning model mainly aims at static graph data, ignores the inherent time attribute of the graph data, and cannot learn the evolution of nodes and the relation thereof in the graph data and update the node embedding, so that the graph representation result lacks dynamics and reality.

In recent years, a dynamic representation learning technology for Graph data is widely developed, and a Graph Convolution neural network (GCN) in a Graph neural network has the characteristics of simple structure, low complexity, less training times, good learning effect and the like, and is applied to a plurality of models as a basis for Graph data representation. The GCN itself cannot capture dynamic features in the graph data, and a Recurrent Neural Network (RNN) can process sequence information containing time relations, which means that the RNN can learn dynamic features of nodes in the graph data over time. Thus, many approaches combine GCN with RNN techniques to construct a dynamic graph representation of learning models, such as GCRN, RgCNN, AddGraph, and the like. The methods input the structural features of the graph data captured by the GCN into the RNN, learn the time sequence relationship of the nodes and capture the dynamic features of the graph data. However, as the graph data evolves, the graph structure changes, and a single GCN model can only capture the currently existing node features, so that the feature learning effect on new nodes is poor. And the parameters of the GCN model are evolved by using RNN according to EverGCN [ Pareja A, Domenioni G, Chen J, et al, EverGCN: evaporating Graph structural Networks for Dynamic Graphs [ C ]// AAAI.2020:5363-5370 ] proposed by Pareja, so that the Graph structural features extracted by the GCN at different times have time sequence. The EvolveGCN belongs to supervised learning, and can not apply unlabeled data, particularly clustering, community discovery and other tasks, so that the node embedding result is poor in universality.

Disclosure of Invention

The invention provides a method for constructing a double-layer evolved dynamic graph convolution neural network model and application aiming at the problems of poor characteristic learning effect on new nodes and poor node embedding result universality in the dynamic representation learning of the existing graph data.

In order to achieve the purpose, the invention adopts the following technical scheme:

the invention provides a method for constructing a double-layer evolved dynamic convolutional neural network model, which is based on a convolutional neural network (GCN), utilizes a Recurrent Neural Network (RNN) to perform double-layer evolution on network parameters and node embedding of the GCN respectively, captures the change of the structural characteristics of graph data, ensures the stability of the node embedding in time, constructs a loss function by utilizing a node embedding evolution result and a result generated by the GCN, corrects the GCN network parameters, forms an unsupervised dynamic learning graph representation frame and completes construction of a double-layer evolved dynamic convolutional neural network model (DEDGCN).

Furthermore, the GCN consists of 2 graph convolution layers, and the structural characteristics of the nodes in the network are extracted on the basis of the characteristics of the neighbor nodes; for graph G_t＝(V_t,E_t)，V_tRepresenting the set of nodes under the snapshot at time t, E_tRepresenting the set at the lower edge of the snapshot at time t, the propagation rules between graph convolution layers are as follows:

A′_t＝A_t+I

wherein H_t ^(l)Represents the result obtained after the t-time graph data is convoluted by the l-th layer graph,

adjacency matrix A representing t-time graph data_tOf regularized form, A'_tSelf-join matrix representing t-time graph data, D degree matrix, W_t ^(l)Represents the parameter of the ith layer graph convolution layer at the time t, and sigma (DEG) represents an activation function;

the input of the GCN is a vector of a node in the t-moment graph data, and after the L-layer graph is passed, the output of the GCN is embedded into the node obtained by learning.

Further, the GCN parameter of the l-th layer at the time t-1 is determined

Inputting the signal into an LSTM to obtain a GCN parameter W of the l-th layer at the time t_t ^(l)The GCN parameter evolution calculation method comprises the following steps:

wherein i^(t)Indicating the gate calculation at time t, U⁽ⁱ⁾Representing the weight parameter of the input gate, B⁽ⁱ⁾Indicating the offset parameter of the input gate, f^(t)Indicating the forgetting to remember the calculation result at time t, U^(f)Weight parameter representing forgetting to remember gate, B^(f)Offset parameter representing forgetting to remember gate, o^(t)Represents the output gate calculation at time t, U^(o)Representing the weight parameter of the output gate, B^(o)A bias parameter indicative of the output gate,

represents the candidate cell calculation at time t, U^(c)Representing the weight parameter of the candidate cell, B^(c)Representing candidate detailsBias parameter of the cell, c^(t)Shows the result of cell renewal at time t, c^(t-1)Represents the cell update result at time t-1, and tanh () represents the activation function.

Further, nodes obtained through GCN at different time are embedded into { X_t-w+1,X_t-w+2,…,X_tInputting the data into the LSTM to predict the embedding of the node at the next moment, wherein the GCN node evolution is calculated as follows:

P_t+1＝LSTM(X_t,P_t)

i^(t+1)＝sigmoid(W⁽ⁱ⁾x^(t)+U⁽ⁱ⁾P_t+B⁽ⁱ⁾)

f^(t+1)＝sigmoid(W^(f)x^(t)+U^(f)P_t+B^(f))

o^(t+1)＝sigmoid(W^(o)x^(t)+U^(o)P_t+B^(o))

where w represents the window size of the LSTM, P_tNode embedding at time t, i, representing a prediction^(t+1)Represents the gate calculation result at time t +1, W⁽ⁱ⁾、U⁽ⁱ⁾Representing the weight parameter of the input gate, B⁽ⁱ⁾Indicating the offset parameter of the input gate, f^(t+1)Indicating that the gate calculation result was forgotten at time t +1, W^(f)、U^(f)Weight parameter representing forgetting to remember gate, B^(f)Offset parameter representing forgetting to remember gate, o^(t+1)Represents the output gate calculation result at time t +1, W^(o)、U^(o)Weight parameter representing output gateNumber, B^(o)A bias parameter indicative of the output gate,

represents the calculation result of candidate cells at time t +1, W^(c)、U^(c)Representing the weight parameter of the candidate cell, B^(c)Representing a bias parameter of the candidate cell, c^(t+1)Represents the result of cell renewal at time t +1, c^(t)Represents the cell update result at time t, and tanh () represents the activation function.

Further, the loss function is:

where loss represents the loss function and α and β represent the loss function, respectively

And

p represents predicted node embedding, X represents node embedding, and W represents weight parameters in the LSTM model.

The second aspect of the invention provides a node classification method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution, which comprises the following steps:

and (3) learning the node characteristics with the labels through the DEDGCN, and performing probability calculation on node embedding through a feedforward neural network and a softmax function to judge the type of the node u at the time t.

The third aspect of the invention provides an edge classification method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution, which comprises the following steps:

the edge characteristics with the labels are learned through the DEDGCN, and probability calculation is carried out on node embedding through a feedforward neural network and a softmax function, so that the type of the edge (u, v) at the time t is judged.

The fourth aspect of the invention provides a link prediction method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution, which comprises the following steps:

the embedded information of the node u and the node v at the time t and before is calculated through the DEDGCN, whether the edge (u, v) at the time t +1 exists or not is predicted by using the embedded information of the node u and the node v at the time t and before, the characteristics of the node u and the node v are aggregated, and then the existence probability of the edge is obtained by using a feedforward neural network.

Compared with the prior art, the invention has the following beneficial effects:

the invention provides a double-layer evolution dynamic graph convolution neural network model DEDGCN, wherein double-layer evolution is carried out on GCN network parameters and node embedding by using RNN on the basis of a GCN model, the GCN is ensured to capture the constantly changing graph data structure characteristics, the node embedding evolution result is fed back to the GCN network, the GCN network parameters are corrected, and the model learns the node dynamic characteristics and simultaneously captures the stability characteristics. On the other hand, the model constructs a loss function by using the stability characteristics of the nodes and the structural characteristics of the graph, so that the model can be applied to unsupervised tasks and is suitable for application tasks of graph data under various scenes.

Drawings

FIG. 1 is a schematic diagram of a framework of a double-layer evolved dynamic graph convolutional neural network model DEDGCN according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of the evolution process of GCN parameters according to the embodiment of the present invention;

FIG. 3 is a schematic diagram of a node evolution process according to an embodiment of the present invention;

FIG. 4 shows F1 values for node classification tasks on Elliptic data sets for different models;

FIG. 5 shows the F1 values for different models classifying tasks on the Reddit Hyperlink Network and Bitcion Alpha datasets.

Detailed Description

The invention is further illustrated by the following examples in conjunction with the accompanying drawings:

first, we formally define the basic concepts and related problems of dynamic graph representation learning.

G ═ (V, E) denotes a graph, and is composed of a set of nodes V ═ V₁,....,v_NConnection relation (edge) between E ∈ { (v) } and nodes_i,v_j)|(v_i,v_j) E.v × V, where N represents the number of nodes. A represents the adjacency matrix of the graph with the size of N, wherein when e_ij∈E，A _ij1, otherwise A _ij0. If the figure is an undirected graph, A is a symmetric matrix. The dynamic graph is composed of a series of continuously evolving graphs, using

Show, each graph G_t＝(V_t,E_t) Referred to as snapshots. Where t represents the time corresponding to the snapshot, V_tRepresenting the set of nodes under the snapshot at the t-th moment, E_tSet representing the lower edge of the snapshot at time t, A_tThe adjacency matrix of the graph under the snapshot at t moments is shown. In general, we define the problems associated with our work as follows.

Definition 1: (drawing shows learning)

For a given graph G ═ (V, E), graph representation learning is defined as a function

The nodes in the graph are mapped into a d-dimensional vector space. Where d < | V |. Wherein the function f can reserve the structure attribute in the network and the matrix for mapping result

And (4) showing. Simply put, the actual situation is savedThe closer the relationship of the points, the smaller the embedding spatial distance of the nodes.

Definition 2: (dynamic graph representation learning)

For a given dynamic graph

The motion map representation learning is defined as a learning function set F ═ F₁,f₂,…,f_TWill dynamic graph

The nodes in (1) are mapped into d-vector space. Wherein

Mapping function representing t network snapshot, capable of converting node set V of t snapshot_tAll the nodes in the system are mapped into a d vector space, and the similarity of the nodes on two dimensions of structure and time is ensured.

A double-layer evolvement dynamic graph convolution neural network model building method is based on GCN, double-layer evolvement is carried out on GCN network parameters and node embedding through a cyclic neural network, changes of graph data structure characteristics are captured, meanwhile, stability of node embedding in time is guaranteed, and the state of the nodes cannot change greatly in a period of time. A loss function is constructed by using a node embedded evolution result and a result generated by GCN, GCN network parameters are corrected, an unsupervised dynamic graph representation learning frame is formed, and construction of a double-layer evolved dynamic graph convolution neural network model DEDGCN is completed. The DEDGCN construction method mainly comprises three parts of a graph convolution neural network GCN, double-layer evolution and GCN parameter correction, wherein the double-layer evolution comprises GCN parameter evolution and node evolution, and a DEDGCN frame graph is shown in figure 1. It should be noted that, for convenience of description, it is assumed that the number of nodes included in the network at each time is N, but the number of nodes in the network at each time changes with time and is not completely the same, and needs to be distinguished in an actual application process.

(1) Graph convolution neural network GCN

For graph data representation learning techniques, we use the GCN model as the subject for two reasons:

and 1, the GCN model can extract good graph data features without training. Although the GCN belongs to supervised learning, the good capability of extracting the graph data features makes the GCN completely suitable for the unsupervised learning.

And 2, the GCN network has a simple structure. The GCN network usually has good effect by taking 2 to 3 layers, so that the parameters of the model are fewer, and the complexity of time evolution of the parameters is lower.

Generally, the GCN is composed of 2 graph convolution layers, and extracts the structural features of the nodes in the network based on the features of the neighboring nodes. For graph G_t＝(V_t,E_t)，V_tRepresenting the set of nodes under the snapshot at time t, E_tRepresenting the set at the lower edge of the snapshot at time t, the propagation rules between graph convolution layers are as follows:

A′_t＝A_t+I

here, H_t ^(l)Represents the result obtained after the t-time graph data is convoluted by the l-th layer graph,

adjacency matrix A defined as t-time graph data_tOf regularized form, A'_tIs a self-join matrix for t-time plot data, D is a degree matrix,

W_t ^(l)represents the parameter of the layer convolution layer of the ith layer map at time t. σ (-) refers to an activation function, such as a ReLU function or the like.

GCN input H_t ⁽⁰⁾＝X_tThe vectors of the nodes in the t-moment graph data are obtained through an initialization rule. After L layers of graph volume, the output of GCN is H_t ^(L)And indicates the learned node embedding. As can be seen from FIG. 1, for time t, the graph data is node-embedded through two graph convolutional layers.

(2) Two-layer evolution

With the evolution of graph data, the structural features of the graph are changed continuously, and a single GCN model cannot capture the dynamic changes, so that the network is required to have certain adaptivity, and the dynamic structural feature actions can be extracted according to the evolution of the graph. Moreover, the nodes of the dynamic graph data have stability, and the characteristics do not change dramatically within a period of time, which requires that the GCN can adaptively extract the structural characteristics of the dynamic graph and ensure the stability of the nodes. Because the recurrent neural network can capture dynamic characteristics in a time sequence relation and has good prediction and evolution capabilities, the recurrent neural network is used for evolving parameters of the GCN, so that the GCN is adaptively changed along with time change, and dynamic graph structure characteristics are extracted. Meanwhile, a node embedding result is evolved by using a recurrent neural network, the stability characteristics of the node are captured and fed back to the GCN network, and GCN parameters are corrected, so that the GCN can learn the stability characteristics of the node.

Evolution of GCN parameters

At present, a commonly used RNN Memory cell is a Long-Short Term Memory (LSTM) cell, and is composed of an input gate, a forgetting gate and an output gate, and can selectively capture information of a time sequence, store key information and forget redundant information. For the parameter evolution and node evolution of GCN, we select LSTM as memory cell to evolve GCN parameter, and the flow is shown in FIG. 2. The GCN parameter of the l layer at the t-1 moment

Inputting the signal into an LSTM to obtain a GCN parameter W of the l-th layer at the time t_t ^(l)。

The calculation method of GCN parameter evolution is shown as the following formula (1):

represents the candidate cell calculation at time t, U^(c)Representing candidate cellsWeight parameter, B^(c)Representing a bias parameter of the candidate cell, c^(t)Shows the result of cell renewal at time t, c^(t-1)Represents the cell update result at time t-1, and tanh () represents the activation function.

In the process of GCN parameter evolution, the GCN parameter at the t-1 moment has historical information of GCN in the past period, and is further maintained, learned and generated through LSTM cells to form a new GCN parameter containing dynamic characteristics, so that the dynamic characteristics among all snapshots are ensured, and the structural characteristics of dynamic change of graph data can be effectively captured.

b. Node evolution

The evolution of the GCN parameters is only to correlate the dynamics of the graph data from the perspective of a model, and the dynamics of the nodes have a direct effect on the representation of the graph data, namely, the embedding quality of the nodes is directly influenced by the change of the nodes along with time. Therefore, the LSTM is used for extracting the dynamic characteristics of the nodes, the stability characteristics of the nodes are further enhanced, and a basis is made for correcting network parameters.

By capturing the node behaviors in a period of time, the due behaviors of the nodes at the next moment can be predicted, and when the predicted node embedding is similar to the real node embedding, the node embedding learned by the GCN model can directly capture the self dynamic characteristics of the nodes. Because the LSTM has the functions of selective memory and selective memory loss and can predict time sequence data, nodes obtained by GCN at different times are embedded into { X_t-w+1,X_t-w+2,…,X_tThe input is into the LSTM to predict the embedding of the node at the next time, as shown in fig. 3. Where w denotes the window size of the LSTM, x^(t)Indicating the node embedding calculated at time t, P_tThe predicted node embedding at time t is expressed by the following formula (2):

P_t+1＝LSTM(X_t,P_t)

i^(t+1)＝sigmoid(W⁽ⁱ⁾x^(t)+U⁽ⁱ⁾P_t+B⁽ⁱ⁾)

f^(t+1)＝sigmoid(W^(f)x^(t)+U^(f)P_t+B^(f))

o^(t+1)＝sigmoid(W^(o)x^(t)+U^(o)P_t+B^(o))

i^(t+1)represents the gate calculation result at time t +1, W⁽ⁱ⁾、U⁽ⁱ⁾Representing the weight parameter of the input gate, B⁽ⁱ⁾Indicating the offset parameter of the input gate, f^(t+1)Indicating that the gate calculation result was forgotten at time t +1, W^(f)、U^(f)Weight parameter representing forgetting to remember gate, B^(f)Offset parameter representing forgetting to remember gate, o^(t+1)Represents the output gate calculation result at time t +1, W^(o)、U^(o)Representing the weight parameter of the output gate, B^(o)A bias parameter indicative of the output gate,

(III) GCN parameter correction

A loss function is constructed by using the stability of nodes, node evolution and a regular form, a GCN network is fed back, and GCN parameters are corrected. The three sections are explained below.

In a dynamic network, the nodes have stability, and the characteristics of the nodes do not change drastically in a period of time, so that the embedding of the nodes between adjacent snapshots is similar as much as possible, and the inherent characteristics of the nodes do not change drastically even in a frequently-increased and-decreased dynamic graph network. Therefore, the constructive loss function is represented by calculating the distance of embedding of the neighboring snapshot nodes, as shown in equation (3).

In the node evolution process, the LSTM is used for predicting the node, a node-embedded future form is constructed, the predicted value of the node embedding is consistent with the actual value of GCN learning, the similarity of the node embedding and the actual value of the GCN learning is calculated, a GCN network is fed back, and the evolution capability of the GCN is improved. The calculation method is as formula (4).

Where P represents predicted node embedding and X represents node embedding.

To prevent the over-fitting phenomenon during the learning process, we add an L2 regular expression to the loss function to reduce the weight. As shown in equation (5), where W represents the weight parameter in the LSTM model.

The final loss function loss is defined as follows.

Wherein alpha and beta represent each

And

the weight of (c).

To verify the effect of the present invention, the DEDGCN was tested in a publicly available basic data set.

Data set 1: elliptic is a bitcoin transaction graph, in which nodes represent transactions and edges represent bitcoin streams between transactions. The attributes of the nodes in the graph are classified into legal and illegal categories, so that the data set is used for carrying out node classification and node clustering tasks in experiments.

Data set 2: bitcoin Alpha is a platform for trading with Bitcoin, and graph data is formed by using trading behaviors among users of the platform. Members of the platform score other members on a scale of-10 to +10 to indicate the degree of trust of each member. Based on the scores, we classify the scores into two categories, trusty and untrustworthy, based on the trust scores of the users. The transaction behaviors among the users are constructed into a dynamic network, the nodes represent the users, the edges represent the transactions among the users, and the scores of the transaction processes of each user are endowed with trusted or untrusted labels. We perform an edge classification task and a link prediction task on the graph dataset.

Data set 3: the Reddit Hyperlink Network extracts a graph formed by the link relations in the Reddit posts, wherein each link relation comprises a time attribute and the emotion (positive or negative) of the source post to the target post. We classify the data set edge-wise and predict the emotional relationships that exist between the unlinked posts.

Data set 4: the UCI is a diagram composed of private information sent by UC-Irvine on the school social platform. On the social platform, a user may search for others and then send a conversation based on profile information. The edge (u, v, t) represents a piece of private information that user u sends to user v at time t. We make a link prediction on the dataset to capture the contacts that the user is likely to have at the next time.

Data set 5: AS Network refers to a graph formed by connections between autonomous systems. The nodes represent autonomous systems, and the edges represent communication relationships between the autonomous systems. We perform link prediction on this data to predict the connectivity of the network at the next time.

The basic information of these data is shown in table 1, and we divide them into snapshots of different time intervals for the characteristics of different data sets and apply them to different tasks.

TABLE 1 figure data information

We will compare with the 5 basic methods below.

The method comprises the following steps: the GCN is a static graph convolution neural network, dynamic features cannot be extracted, all snapshots are fused to form static graph data, and the significance of time features is compared and tested.

The method 2 comprises the following steps: the GCN-GRU is a method for generating network node embedding by using fixed GCN and evolving a node time sequence relation by using GRU.

The method 3 comprises the following steps: DynGEM is a dynamic unsupervised node representation method based on a depth self-coding model, and the model at each moment is trained through evolving self-coding model parameters.

The method 4 comprises the following steps: dynagraph2vec is a dynamic unsupervised node representation method fused with self-coding models, LSTM and MLP, which includes three versions, dynagraph2vecAE, dynagraph 2vecRNN and dynagraph2 vecanern, dynagraph2vecAE are similar in architecture to dynagem, so we compare using the second and third methods.

The method 5 comprises the following steps: the evolveGCN is similar to the GCN-GRU, and the RNN is used for evolving GCN parameters and learning the time sequence relation of the model.

As one possible embodiment, the evaluation methods used in the experiment include three F1 values, MAP, and MRR, and the calculation of these three evaluation methods is described in detail below.

First, we need to define as follows:

true Positive (TP): the predicted value is positive and the true value is the number of samples that are positive.

True Negatives (TN) is the number of samples whose predicted value is positive and the True value is negative.

False Positives (FP), the number of samples for which the predicted value is negative and the true value is positive.

False Negatives (FN), the number of samples whose predicted value is negative and the true value is negative.

With the above 4 definitions, we can calculate the accuracy (Precision) and Recall (Recall) of the prediction results, as follows:

the F1 value is a harmonic mean of the accuracy and the recall ratio, and can objectively reflect the effectiveness of the prediction, and the calculation formula is as follows:

AP refers to the integral of the PR (Precision-Recall) curve, i.e., the average of Precision of all recycle values between 0 and 1. The calculation formula is as follows:

the MAP is an average value of APs of all classes, and is calculated by the following formula, where K represents the number of classes of APs.

Specifically, the present invention verifies the validity of the DEDGCN proposed herein in four aspects of node classification, edge classification, and link prediction.

1. And (3) node classification: and learning the node characteristics with the labels through the DEDGCN, and predicting the types of the nodes without the labels. In the embodiment, probability calculation is performed on node embedding through a feedforward neural network and a softmax function, so as to judge the type of the node u at the time t. For node classification, the present embodiment measures the effectiveness of the method using the F1 value.

2. And (4) edge classification: and learning the edge characteristics with the label through the DEDGCN, and predicting the type of the edge without the label. The present embodiment performs probability calculation on node embedding by using a feedforward neural network (MLP) and a softmax function to determine the type of the edge (u, v) at time t. The characteristics of the edges (u, v) are obtained by adopting the characteristics of the aggregation nodes u and v, and the aggregation method adopts a Hadamard product form. The metric method for edge classification uses the F1 value.

3. And (3) link prediction: embedded information of the node u and the node v at the time t and before is calculated by the DEDGCN, and whether an edge (u, v) at the time t +1 exists or not is predicted by using the embedded information of the node u and the node v at the time t and before. We aggregate the characteristics of node u and node v and then use MLP to obtain the probability of existence of an edge. For link prediction, we use map (mean Average prediction) for the metric.

Specifically, the experimental details are as follows:

1. for any data set, one-hot node-tree is used as an input feature for the model.

2. For all GCNs we set the number of layers to 2; for the MLP used in all classification tasks, the number of layers is set to be 2, classification probability is calculated by utilizing a softmax function, and for the classification tasks and the link prediction tasks, cross entropy is adopted as a loss function of the MLP.

3. For any data set, we use fixed-size embedding as the representation of the node, which is 100.

4. And (3) the training set, the verification set and the test set are adjusted according to the following steps of 8: 1: a scale of 1 divides the data.

Specifically, the experimental results and analyses are as follows:

(1) node classification

We use the Elliptic data set to perform the node classification task, and for both GCN parameters and node embedding evolution, we use a time window size of 5. The result of the node classification is shown in fig. 4.

From fig. 4, we can see that the static graph convolution neural network has a poor representation effect on dynamic graph data, and the F1 value in the three-classification task is only 47%, while the two versions, H version and O version of the evolgcn have a poor node classification effect on the eliptic data set, and particularly the F1 value of the node classification is 44% in the H version. The reason is that the attribute of the node on the Elliptic is relatively stable, mutual conversion between a legal node and an illegal node is difficult to carry out, and the GCN-GRU model and the DEDGCN model proposed by us can evolve the node, extract the inherent characteristics of the node and keep the stability of the node. In addition, the F1 value of the DEDGCN which utilizes MLP to carry out supervised classification reaches 77%, except the supervised classification, the clustering algorithm K-means is adopted to divide the types of the nodes, the effect is only 1% lower than that of the supervised method, and the good effect of the DEDGCN on the unsupervised clustering task is also shown.

(2) Edge classification

Two data sets of Bitcoin Alpha and Reddit Hyperlink Network are used for edge classification tasks, and for GCN parameter and node embedding evolution, the size of a time window adopted by each data set is 5. The edge classification results are shown in fig. 5.

As shown in FIG. 5, the DEDGCN in the edge classification task reaches 93% of F1 value on the Bitcoid Alpha data set and 90% of F1 value on the Reddit Hyperlink Network data set, and meanwhile, the F1 value of EverGCN is higher than that of GCN and GCN-GRU. In graph data, nodes form a graph by means of edges, the edges are important components of a graph structure, the characteristics of the edges mainly depend on the graph structure, in the evolveGCN and the DEDGCN, the graph characteristics are dynamically captured by means of evolution of GCN parameters, the graph structure is accurately grasped, and therefore a good effect is achieved in an edge classification task. The DEDGCN considers the evolution rule of the nodes, and can grasp the development trend and the interaction relation of the nodes, so that the DEDGCN has a better effect in the edge classification task.

(3) Link prediction

We performed link prediction experiments on the three datasets Bitcoin Alpha, UCI and AS Network respectively, and the results are shown in Table 2.

TABLE 2 chaining predicted task MAP values

As can be seen from table 2, the MAP values of DEDGCN are all greater than those of other methods, and are kept at 0.15 or more, which is effective in the link prediction task. The MAP value is the comprehensive reflection of the accuracy and the recall rate and can reflect the overall performance index of the algorithm. The three data sets of Bitcoin Alpha, UCI and AS Network respectively belong to three different types of graph data of a transaction Network, a social Network and an equipment Network, and the DEDGCN has stable MAP values on the different types of graph data, which directly shows that the DEDGCN can be suitable for most graph data and has better robustness.

The effect of our proposed DEDGCN is demonstrated by its performance on node classification, edge classification and link prediction tasks. Based on experimental results, a task scene applicable to two conditions of GCN-RNN based on node evolution and EvloveGCN based on parameter evolution is discussed, and DEDGCN combines the advantages of node evolution and parameter evolution, so that the extracted node characteristics have dynamic property, the inherent stability characteristics of the node are kept, good effects are obtained in various tasks, and the applicability of the DEDGCN is proved. In addition, the DEDGCN belongs to an unsupervised graph representation model, the node type is judged by using an unsupervised clustering method in a node classification task according to experimental data, the effect is only 1% lower than that of the supervised classification task, and the effectiveness of the DEDGCN in the unsupervised task is also shown.

In summary, the invention provides a double-layer evolution dynamic graph convolutional neural network model DEDGCN, wherein double-layer evolution is carried out on GCN network parameters and node embedding by using RNN on the basis of a GCN model, the GCN is ensured to capture continuously-changing graph data structure characteristics, the evolution result of node embedding is fed back to the GCN, the GCN network parameters are corrected, and the model learns the node dynamic characteristics and simultaneously captures the stability characteristics. On the other hand, the model constructs a loss function by using the stability characteristics of the nodes and the structural characteristics of the graph, so that the model can be applied to unsupervised tasks and is suitable for application tasks of graph data under various scenes.

The above shows only the preferred embodiments of the present invention, and it should be noted that it is obvious to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and these modifications and improvements should also be considered as the protection scope of the present invention.

Claims

1. A double-layer evolvement dynamic graph convolution neural network model construction method is characterized in that a graph convolution neural network GCN is used as a basis, a recurrent neural network RNN is used for carrying out double-layer evolvement on network parameters and node embedding of the GCN respectively, changes of graph data structure characteristics are captured, meanwhile, the stability of node embedding in time is guaranteed, a loss function is constructed by using node embedding evolvement results and results generated by the GCN, GCN network parameters are corrected, an unsupervised dynamic learning graph representation frame is formed, and construction of a double-layer evolvement dynamic graph convolution neural network model DEDGCN is completed.

2. The method for constructing a double-layer evolved dynamic graph convolution neural network model according to claim 1, wherein the GCN consists of 2 graph convolution layers, and structural features of nodes in the network are extracted on the basis of the features of neighbor nodes; for graph G_t＝(V_t,E_t)，V_tRepresenting the set of nodes under the snapshot at time t, E_tRepresenting the set at the lower edge of the snapshot at time t, the propagation rules between graph convolution layers are as follows:

A′_t＝A_t+I

wherein H_t ^(l)Representing t time graph data through the convolution of the first layer graphThe result obtained after that is that,

3. The method for constructing a double-layer evolved dynamic graph convolution neural network model as claimed in claim 2, wherein the GCN parameter of the l-th layer at the time t-1 is determined

represents the candidate cell calculation at time t, U^(c)Representing the weight parameter of the candidate cell, B^(c)Representing a bias parameter of the candidate cell, c^(t)Shows the result of cell renewal at time t, c^(t-1)Represents the cell update result at time t-1, and tanh () represents the activation function.

4. The method for constructing a two-layer evolved dynamic graph convolution neural network model according to claim 1, wherein nodes obtained through GCN at different times are embedded into { X_t-w+1,X_t-w+2,…,X_tInputting the data into the LSTM to predict the embedding of the node at the next moment, wherein the GCN node evolution is calculated as follows:

P_t+1＝LSTM(X_t,P_t)

i^(t+1)＝sigmoid(W⁽ⁱ⁾x^(t)+U⁽ⁱ⁾P_t+B⁽ⁱ⁾)

f^(t+1)＝sigmoid(W^(f)x^(t)+U^(f)P_t+B^(f))

o^(t+1)＝sigmoid(W^(o)x^(t)+U^(o)P_t+B^(o))

where w represents the window size of the LSTM, P_tNode embedding at time t, i, representing a prediction^(t+1)Represents the gate calculation result at time t +1, W⁽ⁱ⁾、U⁽ⁱ⁾Representing the weight parameter of the input gate, B⁽ⁱ⁾Indicating the offset parameter of the input gate, f^(t+1)Indicating that the gate calculation result was forgotten at time t +1, W^(f)、U^(f)Weight parameter representing forgetting to remember gate, B^(f)Offset parameter representing forgetting to remember gate, o^(t+1)Represents the output gate calculation result at time t +1, W^(o)、U^(o)Representing the weight parameter of the output gate, B^(o)A bias parameter indicative of the output gate,

5. The method of constructing a two-layer evolved dynamic graph convolution neural network model of claim 4, wherein the loss function is:

And

6. A node classification method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution is characterized by comprising the following steps:

7. An edge classification method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution is characterized by comprising the following steps:

8. A link prediction method of a dynamic graph convolutional neural network model DEDGCN based on double-layer evolution is characterized by comprising the following steps: