CN112925953B - Dynamic network representation method and system - Google Patents
Dynamic network representation method and system Download PDFInfo
- Publication number
- CN112925953B CN112925953B CN202110256274.7A CN202110256274A CN112925953B CN 112925953 B CN112925953 B CN 112925953B CN 202110256274 A CN202110256274 A CN 202110256274A CN 112925953 B CN112925953 B CN 112925953B
- Authority
- CN
- China
- Prior art keywords
- sequence
- representation
- node
- edge
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000013598 vector Substances 0.000 claims abstract description 152
- 239000011159 matrix material Substances 0.000 claims abstract description 58
- 230000007246 mechanism Effects 0.000 claims abstract description 22
- 238000004891 communication Methods 0.000 claims abstract description 17
- 238000013507 mapping Methods 0.000 claims description 78
- 230000004927 fusion Effects 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000005295 random walk Methods 0.000 claims description 10
- 238000012935 Averaging Methods 0.000 claims description 7
- 238000011478 gradient descent method Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Algebra (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a dynamic network representation method and a dynamic network representation system. The method comprises the following steps: generating a node sequence of the communication network; constructing a plurality of independent hidden spaces, taking node position information as constraint, and adopting a self-attention mechanism to learn node sequence relation of the node sequence to obtain a first node representation sequence fusing multiple potential relation information among nodes; converting the first node representation sequence into an edge representation sequence, carrying out edge sequence relation learning on the edge representation sequence by taking the timestamp information as constraint and adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges; determining a sequence vector representation from the first edge representation sequence, calculating cross entropy loss, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the communication network. The invention can accurately mine the evolution characteristics of the network structure, thereby improving the accuracy of downstream task services such as link prediction, node classification and the like.
Description
Technical Field
The present invention relates to the field of network data mining, and in particular, to a dynamic network representation method and system.
Background
With the advent of the internet, data mining tasks based on various networks (e.g., social networks, communication networks, collaborative networks, etc.) have become increasingly important, and dynamic networks are a very important tool for representing networks. Network data of various networks is often complex and difficult to process, and its network structure changes in real time, so how to mine dynamic evolution features from historical network structures, learning low-dimensional vector representations of nodes is necessary.
In real life, many networks are dynamic, which can change in structure over time. The dynamic network representation method greatly improves the representation capability of the learned node representation vector by capturing the dynamic change of the network, thereby better serving the downstream task. Network representation learning, also known as network embedding, graph embedding, aims to represent nodes in a network as low-dimensional, real-valued, dense vector forms, so that the resulting vector forms can have the capability of representation and reasoning in vector space, and can be easily and conveniently used as input of a machine learning model, and the resulting vector representations can be further applied to common applications in the network. Network representation learning is an efficient method of mining dynamically evolving features of network structures and learning low-dimensional vector representations of nodes.
Early dynamic network representation methods split networks into multiple static networks (i.e., snapshot) based on successive deadlines, and learn node representations by learning changes between adjacent snapshot. TNE is a non-negative matrix factorization-based dynamic network representation method that employs a non-negative matrix factorization technique on each snapshot to learn vector representations for nodes of all snapshot. Although the nodes have one node representation on all snapshot, they are not independent of each other. The regular term on time smoothing is added to the objective function of matrix decomposition taking into account the time-lapse relationship of adjacent snapshot. Such methods tend to ignore the dynamics in the snapshot, for example, in a certain snapshot, the node pairs are connected first and then disconnected, the information reflected into the snapshot is that no connection is generated between the nodes, and obviously, the learning difference of the two information to the node representation is huge.
In order to solve the problem of massive loss of dynamic information based on the snapshot method, a dynamic network representation method based on sequence learning is proposed. Such methods convert a dynamic network into a chronological sequence of nodes and learn the node representation by different sequence learning methods. CTDNE generates a sequence of nodes by time-domain random walk. But CTDNE learns the sequence using Skip-Gram model, losing the order information contained in the node sequence. Since the Skip-Gram model only concerns the co-occurrence probability of the central node and the context node, it does not concern the order in which the nodes appear. HTNEs improve CTDNE by integrating the position information of nodes in the sequence into the learning process to preserve the order information between nodes. The existing dynamic network representation method based on sequence learning only considers time in the sequence generation stage, but the learning of time information and even node position information is lacking in the subsequent sequence learning model, and the learned node relationship is single and does not consider the diversity of the node relationship in the sequence. Therefore, the existing dynamic network representation method based on sequence learning cannot accurately mine the evolution characteristics of the network structure, so that the accuracy of downstream task services such as link prediction, node classification and the like can be affected.
Disclosure of Invention
Based on this, it is necessary to provide a dynamic network representation method and system to accurately mine the evolution characteristics of the network structure, so as to improve the accuracy of downstream task services such as link prediction and node classification.
In order to achieve the above object, the present invention provides the following solutions:
a dynamic network representation method, comprising:
generating a node sequence of the network;
constructing a plurality of independent hidden spaces, taking node position information as constraint, and adopting a self-attention mechanism to learn node sequence relation of the node sequence to obtain a first node representation sequence fused with multiple potential relation information among nodes;
converting the first node representation sequence into an edge representation sequence, using timestamp information as constraint, and performing edge sequence relation learning on the edge representation sequence by adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges;
determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of positive samples and the sequence representations of negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network.
Optionally, the generating the node sequence of the network specifically includes:
and generating a node sequence of the network by adopting a time domain random walk mode.
Optionally, the constructing a plurality of independent hidden spaces, taking node position information as constraint, and performing node sequence relation learning on the node sequence by adopting a self-attention mechanism to obtain a first node representation sequence fusing multiple potential relation information between nodes, which specifically includes:
randomly initializing to generate a node representation matrix and a position representation matrix;
determining node location information from the node sequence;
determining a second node representation sequence fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix;
constructing a plurality of independent hidden spaces;
for each hidden space, mapping the second node representation sequence by adopting three full-connection layers to obtain a mapping vector of the node; the mapping vector comprises a query vector, a keyword vector and a value vector;
and obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
Optionally, the converting the first node representation sequence into an edge representation sequence, using timestamp information as constraint, and performing edge sequence relationship learning on the edge representation sequence by using a self-attention mechanism to obtain a first edge representation sequence of multiple potential relationship information between fused edges, specifically including:
calculating the average value of vectors of two adjacent nodes in the first node representation sequence to obtain an edge representation sequence;
constructing a time stamp sequence from the time stamp information;
connecting the edge representation sequence with the time stamp sequence to obtain a second edge representation sequence fused with time information;
for each hidden space, mapping the second edge representation sequence by adopting three full-connection layers to obtain the mapping vector of the edge;
and obtaining a first edge representation sequence of multiple potential relation information among the fused edges based on the mapping vector of the edges.
Optionally, the determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of the positive samples and the sequence representations of the negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network, which specifically includes:
averaging the first edge representation sequence to obtain a sequence vector representation;
for all the sequence vector representations corresponding to the node sequences, calculating cross entropy loss between the sequence representation of the positive sample and the sequence representation of the negative sample; the cross entropy loss is
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping the vector of the ith node sequence to a one-dimensional vector obtained in one dimension;
according to the cross entropy loss, updating learning parameters by adopting a gradient descent method, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
The invention also provides a dynamic network representation system, which comprises:
the node sequence generating module is used for generating a node sequence of the network;
the first sequence relation learning module is used for constructing a plurality of independent hidden spaces, carrying out node sequence relation learning on the node sequence by taking node position information as constraint and adopting a self-attention mechanism to obtain a first node expression sequence fused with multiple potential relation information among nodes;
the second sequence relation learning module is used for converting the first node representation sequence into an edge representation sequence, carrying out edge sequence relation learning on the edge representation sequence by taking the timestamp information as a constraint and adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges;
and the optimal node representation determining module is used for determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of the positive samples and the sequence representations of the negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as the optimal node representation of the network.
Optionally, the node sequence generating module specifically includes:
and the node sequence generating unit is used for generating a node sequence of the network in a time domain random walk mode.
Optionally, the first sequence relation learning module specifically includes:
the initialization unit is used for randomly initializing and generating a node representation matrix and a position representation matrix;
a position information determining unit configured to determine node position information from the node sequence;
the first fusion unit is used for determining a second node representation sequence for fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix;
the hidden space construction unit is used for constructing a plurality of independent hidden spaces;
the node mapping unit is used for mapping the second node representation sequence to obtain the mapping vector of the node by adopting three full-connection layers for each hidden space; the mapping vector comprises a query vector, a keyword vector and a value vector;
and the second fusion unit is used for obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
Optionally, the second sequence relation learning module specifically includes:
the edge representation sequence calculation unit is used for calculating the average value of vectors of two adjacent nodes in the first node representation sequence to obtain an edge representation sequence;
a time stamp sequence construction unit for constructing a time stamp sequence from the time stamp information;
the third fusion unit is used for connecting the edge representation sequence with the time stamp sequence to obtain a second edge representation sequence of fusion time information;
an edge mapping unit, configured to map the second edge representation sequence to obtain a mapping vector of an edge by using three full connection layers for each hidden space;
and the fourth fusion unit is used for obtaining a first edge expression sequence of multiple potential relation information among the fusion edges based on the mapping vectors of the edges.
Optionally, the optimal node representation determining module specifically includes:
a sequence vector representation calculation unit, configured to average the first edge representation sequence to obtain a sequence vector representation;
a loss calculation unit for calculating cross entropy loss between the sequence representation of the positive sample and the sequence representation of the negative sample for the sequence vector representations corresponding to all the node sequences; the cross entropy loss is
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping the vector of the ith node sequence to a one-dimensional vector obtained in one dimension;
the iteration updating unit is used for updating learning parameters by adopting a gradient descent method according to the cross entropy loss, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a dynamic network representation method and a system, which enable the learned node representation to have position rationality and time rationality by learning the position information of the node sequence and the time stamp information of the edge sequence, can accurately mine the evolution characteristics of the network structure, and improve the accuracy of downstream task services such as link prediction, node classification and the like; in node sequence learning and edge sequence learning, sequence correlation is mined by using a self-attention mechanism in a plurality of independent hidden spaces, so that the capability of mining potential diversity relations of the sequences is greatly improved; compared with the sequence learning method based on RNN, the calculation based on the self-attention mechanism does not depend on the output of the last moment, so that the parallelization calculation can be performed, and the time efficiency is higher.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a dynamic network representation method provided by an embodiment of the present invention;
FIG. 2 is a process diagram of a specific implementation of a dynamic network representation method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of comparison of different sequence learning methods in a communication network according to an embodiment of the present invention;
fig. 4 is a block diagram of a dynamic network representation system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
The invention adoptsRepresenting the dynamic network of inputs, i.e. the communication network. V= (V) 0 ,v 1 ,…,v |V| ) Is the set of all nodes of the dynamic network, where |v| represents the number of nodes. E is the set of all edges of the dynamic network. />Is a mapping function, and maintains the mapping relation between each edge and the corresponding time attribute t. The dynamic network representation aims at learning a vector representation of dimension D for each node in the network, where D < |v|.
Fig. 1 is a flowchart of a dynamic network representation method according to an embodiment of the present invention.
Referring to fig. 1, the dynamic network representation method of the present embodiment includes:
step 101: a sequence of nodes of the network is generated. Specifically, a node sequence of the network is generated by adopting a time domain random walk mode. The network may be a social network, a communication network, a collaborative network, or the like.
The node sequence of the network is generated by adopting a time domain random walk mode, and the specific implementation mode is as follows:
time domain random walk is an improved random walk that performs time biased walk in the time domain neighborhood. Suppose from the last node V la Walk to the current node V cu Wherein e=<v la ,v cu >∈E，f(e)＝t。V cu The time domain neighbors at time t are:
Γ t (v cu )＝{v ne |<v cu ,v ne >∈E∧f(<v cu ,v ne >)>t}。
selecting a next hop node v based on a probability of time bias ne ∈Γ t (v cu ). The probability of time bias is defined as:
wherein the method comprises the steps ofIs v la And v ne The distance of the shortest path between the two paths, P is the return parameter, q is the ingress and egress parameter, P t (v la ,v cu ,v ne T) is the time transition probability, which is defined as follows:
at maximum length K max Under the limit of (1), two connected nodes are randomly selected as initial V la And V cu Selecting the next hop node V according to the definition ne . Thereafter V cu V is changed la ，V ne V is changed cu And continuing to walk in the same way, and adding the traversed nodes into the node sequence according to the walking order. Multiple wandering times to obtain multiple wandering timesA sequence of nodes as input for subsequent sequence learning of nodes.
Step 102: and constructing a plurality of independent hidden spaces, taking node position information as constraint, and adopting a self-attention mechanism to learn node sequence relation of the node sequence to obtain a first node representation sequence fused with multiple potential relation information among nodes.
Step 102 is to learn the sequence relationship between nodes under the constraint that the node sequence learns the position information of the nodes. Assume that the length of the input sequence is L s By usingAnd (3) representing. The method specifically comprises the following steps:
1) To incorporate the location information of the nodes into the learning process, a learnable representation matrix is maintained for the discrete location variables. First randomly initializing to generate node representation matrix H V And a position representation matrix H P Wherein H is V ∈R |V|*D ，
2) Node location information is determined from the sequence of nodes. The position information of S can be expressed as p= [0,1, …, L s -1]。
3) And determining a second node representation sequence fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix.
Specifically, S and P are respectively input into a node representation matrix and a position representation matrix to obtain a node representation sequence S V And a position representation sequence S P ：
S V ＝LookUp(H V ,S)，
S P ＝LookUp(H P ,P)，
Wherein the method comprises the steps ofLookUp is a function of looking up vectors from a matrix by subscripts. Is thatCombining learning relation information and position information, and combining S V And S is P And (3) connecting:
wherein the method comprises the steps ofS' V Is a node representation sequence, i.e. a second node representation sequence, fused with node location information.
4) To learn multiple potential relationships between nodes, self-attention mechanisms are used in m independent hidden spaces. The method comprises the following steps:
a plurality of independent hidden spaces are constructed. For each hidden space, mapping the second node representation sequence by adopting three full-connection layers to obtain a mapping vector of the node; the mapping vector includes a query vector, a keyword vector, and a value vector. And obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
For example, in the ith (0.ltoreq.i)<m) in the hidden spaces, S 'is formed by three full-connection layers' V Respectively mapped into query vectors Q i Keyword vector K i Sum vector V i Wherein Q is i ＝S' v *W Qi +b Qi ，K i ＝S' v *W Ki +b Ki ，V i ＝S' v *W Vi +b Vi . Using the formulaCalculating a relation weight between any two node representations in the sequence, wherein D represents the dimension of a vector representation learned by each node in the network, T represents transposition and W Qi And b Qi Weights and biases of query vectors representing nodes, W Ki And b Ki Weights and biases of keyword vectors representing nodes, W Vi And b Vi The weights and offsets of the value vectors representing the nodes. By means ofOther nodes in this weight fusion sequence represent get +.>Finally->From m->Averaging, thus->Multiple potential relationship information between nodes is fused, namely, a first node represents a sequence. The algorithm implementation process is as follows:
step 103: and converting the first node representation sequence into an edge representation sequence, and carrying out edge sequence relation learning on the edge representation sequence by taking the timestamp information as constraint and adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges.
Step 103 is to learn the sequence relationship between edges under the constraint of the edge sequence learning under the time information. The method specifically comprises the following steps:
1) The vector representation of the edge connecting two adjacent nodes in the sequence of nodes may be obtained by averaging the vector representations of the two nodes. Thus, the average value of the vectors of two adjacent nodes in the first node representation sequence is calculated to obtain the edge representation sequence, and thus the first node representation sequenceConversion to the edge representation sequence +.>
2) When constructed from the time stamp informationA sequence of interstags. The time stamp is a real number, and the time stamp sequence is used for containing the time stamp information on the edgeIs turned into->
S T ＝T*w t ，
Wherein w is t ∈R 2D Is a gaussian vector of 0 mean.
3) To use the time stamp as constraint of the learning of the edge sequence relation, the edge is represented as a sequence S E And the time stamp sequence S T Connecting to obtain a second side representation sequence of the fusion time information
Wherein the method comprises the steps of
4) To learn multiple sequence relationships between edges, a self-attention mechanism is used in m independent hidden spaces. For each hidden space, mapping the second edge representation sequence by adopting three full-connection layers to obtain the mapping vector of the edge; and obtaining a first edge representation sequence of multiple potential relation information among the fused edges based on the mapping vector of the edges.
For example, in the ith (0.ltoreq.i)<m) in the hidden spaces, S 'is formed by three full-connection layers' E Respectively mapped to query vectors Q1 i Keyword vector K1 i Sum vector V1 i Wherein Q1 i ＝S' E *W1 Qi +b1 Qi ，K1 i ＝S' E *W1 Ki +b1 Ki ，V1 i ＝S' E *W1 Vi +b1 Vi . Using the formulaCalculating to obtain a relation weight1 and W1 between any two node representations in the sequence Qi And b1 Qi Weights and offsets of query vectors representing edges, W1 Ki And b1 Ki Weights and biases of keyword vectors representing edges, W1 Vi And b1 Vi The weights and offsets of the value vectors representing the edges. Using other node representations in the weight fusion sequence Finally->From m->Averaging, thus->Multiple pieces of potential relationship information between edges are fused, i.e., the first edge represents a sequence. The algorithm implementation process is as follows:
step 104: determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of positive samples and the sequence representations of negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network.
The step 104 specifically includes:
1) The sequence represents an effect assessment.
(1) Representing a sequence for the first edgeAveraging to obtain a sequence vector representation.
(2) For all the sequence vector representations corresponding to the node sequences, a cross entropy penalty between the sequence representation of positive samples and the sequence representation of negative samples is calculated. The sequential representation of positive samples and the sequential representation of negative samples should be separated as far as possible in the vector space. The more separated positive and negative samples are in the vector space, the better the node representation effect of the sequence is. To measure the position of a sequence in vector space, a high-dimensional vector representation of the ith sequence is mapped to a one-dimensional variable delta using a fully connected layer i ∈[0,1]Representing the position in one dimension. Delta of positive sample i The value should be close to 1, delta for negative samples i The value should be close to 0. Cross entropy loss is used to measure the degree of separation, and the cross entropy loss is calculated by the formula
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping of the vector of the i-th node sequence to a one-dimensional vector obtained in one dimension.
2) Node representation optimization
According to the cross entropy loss, updating learning parameters by adopting a gradient descent method, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
In practical application, after the performance metrics are represented by the sequences, the learning parameters theta of the model are optimized by using the Adam algorithm of the gradient descent method, and the learning parameters theta comprise two representation matrices H V And H P Weight W * And deviation b * Weight W1 * And deviation b1 * . To prevent model overfitting, an early stop technique is employed. If the cross entropy loss stops dropping, the model will wait for several rounds. If the loss has not continued to drop after the waiting round, the optimization stops and the node is considered to represent no further optimization.
A specific implementation process of the dynamic network representation method of this embodiment is shown in fig. 2.
The advantages of the dynamic network representation method of the present embodiment are described below.
For example, in a communication network, as shown in fig. 3, each node is a person, and if M calls N at time t, an edge is generated between M and N in the communication network, and the time attribute of the edge is t. From this network a length 3 sequence was run out, containing Allen, carl and Ben.
For this sequence, the information learned by the existing sequence learning method is different, for example, the Skip-Gram model used by CTDNE only learns the relationship information (Correlation) between people, namely: allen and Carl, ben have been telephonically past, indicating that Allen is simultaneously aware of Carl and Ben, as shown in section A of FIG. 3. And the HTNE further learns the Position information (correlation+position) of each person on the sequence based on the relation information, namely: carl calls Allen first, then Allen calls Ben again, as shown in part B of FIG. 3. The method of this embodiment further includes the edge time information into learning (correlation+position+time), that is: carl calls Allen at 2019.1, then Allen calls Ben at 2020.9, as shown in part C of FIG. 3. Therefore, the information in the sequence is learned more comprehensively, and the relation information, the position information and the time information are contained. The diversity learning of relationships is based on the consideration that telephone communication between two persons may be job site communication, private friend communication, or even other types of relationships, and existing methods basically consider learning of only a single relationship, and the method of the embodiment increases learning of multiple relationships through the assumption of multiple hidden spaces.
The present invention also provides a dynamic network representation system, referring to fig. 4, the dynamic network representation system of the present embodiment includes:
a node sequence generating module 201, configured to generate a node sequence of the network.
The first sequence relation learning module 202 is configured to construct a plurality of independent hidden spaces, perform node sequence relation learning on the node sequence by using node position information as constraint and adopting a self-attention mechanism, so as to obtain a first node representation sequence fusing multiple potential relation information between nodes.
The second sequence relation learning module 203 is configured to convert the first node representation sequence into an edge representation sequence, perform edge sequence relation learning on the edge representation sequence by using the timestamp information as a constraint and using a self-attention mechanism, so as to obtain a first edge representation sequence fused with multiple potential relation information between edges.
An optimal node representation determining module 204, configured to determine a sequence vector representation from the first edge representation sequence, calculate cross entropy loss between the sequence representations of positive samples and the sequence representations of negative samples in all the sequence vector representations, and determine a node representation matrix corresponding to when the cross entropy loss stops descending as an optimal node representation of the network.
As an optional implementation manner, the node sequence generating module 201 specifically includes:
and the node sequence generating unit is used for generating a node sequence of the network in a time domain random walk mode.
As an optional implementation manner, the first sequence relationship learning module 202 specifically includes:
and the initialization unit is used for randomly initializing and generating a node representation matrix and a position representation matrix.
And the position information determining unit is used for determining the node position information from the node sequence.
And the first fusion unit is used for determining a second node representation sequence for fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix.
And the hidden space construction unit is used for constructing a plurality of independent hidden spaces.
The node mapping unit is used for mapping the second node representation sequence to obtain the mapping vector of the node by adopting three full-connection layers for each hidden space; the mapping vector includes a query vector, a keyword vector, and a value vector.
And the second fusion unit is used for obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
As an optional implementation manner, the second sequence relationship learning module 203 specifically includes:
and the edge representation sequence calculation unit is used for calculating the average value of vectors of two adjacent nodes in the first node representation sequence to obtain an edge representation sequence.
And the time stamp sequence construction unit is used for constructing a time stamp sequence from the time stamp information.
And the third fusion unit is used for connecting the edge representation sequence and the time stamp sequence to obtain a second edge representation sequence of the fusion time information.
And the edge mapping unit is used for mapping the second edge representation sequence to obtain the mapping vector of the edge by adopting three full-connection layers for each hidden space.
And the fourth fusion unit is used for obtaining a first edge expression sequence of multiple potential relation information among the fusion edges based on the mapping vectors of the edges.
As an optional implementation manner, the optimal node representation determining module 204 specifically includes:
and the sequence vector representation calculation unit is used for averaging the first edge representation sequence to obtain a sequence vector representation.
A loss calculation unit for calculating cross entropy loss between the sequence representation of the positive sample and the sequence representation of the negative sample for the sequence vector representations corresponding to all the node sequences; the cross entropy loss is
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping of the vector of the i-th node sequence to a one-dimensional vector obtained in one dimension.
The iteration updating unit is used for updating learning parameters by adopting a gradient descent method according to the cross entropy loss, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (10)
1. A dynamic network representation method, comprising:
generating a node sequence of the network; the network is a communication network; each node is one of the communication networks, M in the communication network creating an edge between M and N if N is called at time t;
constructing a plurality of independent hidden spaces, taking node position information as constraint, and adopting a self-attention mechanism to learn node sequence relation of the node sequence to obtain a first node representation sequence fused with multiple potential relation information among nodes;
converting the first node representation sequence into an edge representation sequence, using timestamp information as constraint, and performing edge sequence relation learning on the edge representation sequence by adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges;
determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of positive samples and the sequence representations of negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network.
2. The method for dynamic network representation according to claim 1, wherein the generating the node sequence of the network specifically comprises:
and generating a node sequence of the network by adopting a time domain random walk mode.
3. The method for dynamic network representation according to claim 1, wherein the constructing a plurality of independent hidden spaces, using node position information as a constraint, and performing node sequence relation learning on the node sequence by using a self-attention mechanism, to obtain a first node representation sequence fusing multiple potential relation information between nodes, comprises:
randomly initializing to generate a node representation matrix and a position representation matrix;
determining node location information from the node sequence;
determining a second node representation sequence fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix;
constructing a plurality of independent hidden spaces;
for each hidden space, mapping the second node representation sequence by adopting three full-connection layers to obtain a mapping vector of the node; the mapping vector comprises a query vector, a keyword vector and a value vector;
and obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
4. A dynamic network representation method according to claim 3, wherein said converting the first node representation sequence into an edge representation sequence, using the timestamp information as a constraint, and performing edge sequence relationship learning on the edge representation sequence by using a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relationship information between edges, comprises:
calculating the average value of vectors of two adjacent nodes in the first node representation sequence to obtain an edge representation sequence;
constructing a time stamp sequence from the time stamp information;
connecting the edge representation sequence with the time stamp sequence to obtain a second edge representation sequence fused with time information;
for each hidden space, mapping the second edge representation sequence by adopting three full-connection layers to obtain the mapping vector of the edge;
and obtaining a first edge representation sequence of multiple potential relation information among the fused edges based on the mapping vector of the edges.
5. The method according to claim 4, wherein determining a sequence vector representation from the first edge representation sequence, calculating cross entropy loss between the sequence representation of positive samples and the sequence representation of negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to when the cross entropy loss stops dropping as an optimal node representation of the network, specifically comprises:
averaging the first edge representation sequence to obtain a sequence vector representation;
for all the sequence vector representations corresponding to the node sequences, calculating cross entropy loss between the sequence representation of the positive sample and the sequence representation of the negative sample; the cross entropy loss is
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping the vector of the ith node sequence to a one-dimensional vector obtained in one dimension;
according to the cross entropy loss, updating learning parameters by adopting a gradient descent method, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
6. A dynamic network representation system, comprising:
the node sequence generating module is used for generating a node sequence of the network; the network is a communication network; each node is one of the communication networks, M in the communication network creating an edge between M and N if N is called at time t;
the first sequence relation learning module is used for constructing a plurality of independent hidden spaces, carrying out node sequence relation learning on the node sequence by taking node position information as constraint and adopting a self-attention mechanism to obtain a first node expression sequence fused with multiple potential relation information among nodes;
the second sequence relation learning module is used for converting the first node representation sequence into an edge representation sequence, carrying out edge sequence relation learning on the edge representation sequence by taking the timestamp information as a constraint and adopting a self-attention mechanism to obtain a first edge representation sequence fused with multiple potential relation information among edges;
and the optimal node representation determining module is used for determining a sequence vector representation by the first edge representation sequence, calculating cross entropy loss between the sequence representations of the positive samples and the sequence representations of the negative samples in all the sequence vector representations, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as the optimal node representation of the network.
7. The dynamic network representation system according to claim 6, wherein the node sequence generating module specifically comprises:
and the node sequence generating unit is used for generating a node sequence of the network in a time domain random walk mode.
8. The dynamic network representation system according to claim 6, wherein the first sequence relation learning module specifically comprises:
the initialization unit is used for randomly initializing and generating a node representation matrix and a position representation matrix;
a position information determining unit configured to determine node position information from the node sequence;
the first fusion unit is used for determining a second node representation sequence for fusing the node position information according to the node sequence, the node position information, the node representation matrix and the position representation matrix;
the hidden space construction unit is used for constructing a plurality of independent hidden spaces;
the node mapping unit is used for mapping the second node representation sequence to obtain the mapping vector of the node by adopting three full-connection layers for each hidden space; the mapping vector comprises a query vector, a keyword vector and a value vector;
and the second fusion unit is used for obtaining a first node representation sequence fusing multiple potential relation information among the nodes based on the mapping vector of the nodes.
9. The dynamic network representation system according to claim 8, wherein the second sequence relation learning module specifically comprises:
the edge representation sequence calculation unit is used for calculating the average value of vectors of two adjacent nodes in the first node representation sequence to obtain an edge representation sequence;
a time stamp sequence construction unit for constructing a time stamp sequence from the time stamp information;
the third fusion unit is used for connecting the edge representation sequence with the time stamp sequence to obtain a second edge representation sequence of fusion time information;
an edge mapping unit, configured to map the second edge representation sequence to obtain a mapping vector of an edge by using three full connection layers for each hidden space;
and the fourth fusion unit is used for obtaining a first edge expression sequence of multiple potential relation information among the fusion edges based on the mapping vectors of the edges.
10. The dynamic network representation system according to claim 9, wherein the optimal node representation determining module specifically comprises:
a sequence vector representation calculation unit, configured to average the first edge representation sequence to obtain a sequence vector representation;
a loss calculation unit for calculating cross entropy loss between the sequence representation of the positive sample and the sequence representation of the negative sample for the sequence vector representations corresponding to all the node sequences; the cross entropy loss is
Where loss represents cross entropy loss, N represents the total number of node sequences, i represents the ith node sequence, y i Indicating whether the ith node sequence is a positive or negative sample, delta i Representing mapping the vector of the ith node sequence to a one-dimensional vector obtained in one dimension;
the iteration updating unit is used for updating learning parameters by adopting a gradient descent method according to the cross entropy loss, stopping iteration when the cross entropy loss stops descending, and determining a node representation matrix corresponding to the cross entropy loss when the cross entropy loss stops descending as an optimal node representation of the network; the learning parameters include the node representation matrix, the location representation matrix, weights in the mapping vectors of the nodes, deviations in the mapping vectors of the nodes, weights in the mapping vectors of the edges, and deviations in the mapping vectors of the edges.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110256274.7A CN112925953B (en) | 2021-03-09 | 2021-03-09 | Dynamic network representation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110256274.7A CN112925953B (en) | 2021-03-09 | 2021-03-09 | Dynamic network representation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112925953A CN112925953A (en) | 2021-06-08 |
| CN112925953B true CN112925953B (en) | 2024-02-20 |
Family
ID=76172181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110256274.7A Active CN112925953B (en) | 2021-03-09 | 2021-03-09 | Dynamic network representation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112925953B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110334219A (en) * | 2019-07-12 | 2019-10-15 | 电子科技大学 | The knowledge mapping for incorporating text semantic feature based on attention mechanism indicates learning method |
| CN110414665A (en) * | 2019-05-21 | 2019-11-05 | 浙江工业大学 | A kind of network representation learning method based on deep neural network |
| CN111159425A (en) * | 2019-12-30 | 2020-05-15 | 浙江大学 | Temporal knowledge graph representation method based on historical relationship and double-graph convolution network |
| CN111275562A (en) * | 2020-01-17 | 2020-06-12 | 福州大学 | Dynamic community discovery method based on recursive convolutional neural network and self-encoder |
-
2021
- 2021-03-09 CN CN202110256274.7A patent/CN112925953B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414665A (en) * | 2019-05-21 | 2019-11-05 | 浙江工业大学 | A kind of network representation learning method based on deep neural network |
| CN110334219A (en) * | 2019-07-12 | 2019-10-15 | 电子科技大学 | The knowledge mapping for incorporating text semantic feature based on attention mechanism indicates learning method |
| CN111159425A (en) * | 2019-12-30 | 2020-05-15 | 浙江大学 | Temporal knowledge graph representation method based on historical relationship and double-graph convolution network |
| CN111275562A (en) * | 2020-01-17 | 2020-06-12 | 福州大学 | Dynamic community discovery method based on recursive convolutional neural network and self-encoder |
Non-Patent Citations (2)
| Title |
|---|
| Attributed Heterogeneous Network Embedding for Link Prediction;Tingting Wang等;17th Pacific Rim Knowledge Acquisition Workshop, PKAW 2020;20210220;全文 * |
| Attributed Network Embedding for Learning in a Dynamic Environment;Jundong Li等;CIKM’17;20171130;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112925953A (en) | 2021-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111159425B (en) | Temporal knowledge graph representation method based on historical relationship and double-graph convolution network | |
| CN112418482B (en) | Cloud computing energy consumption prediction method based on time series clustering | |
| CN111625361A (en) | Joint learning framework based on cooperation of cloud server and IoT (Internet of things) equipment | |
| Zhang et al. | A short-term traffic forecasting model based on echo state network optimized by improved fruit fly optimization algorithm | |
| Rusek et al. | Message-passing neural networks learn little’s law | |
| CN112184391A (en) | Recommendation model training method, medium, electronic device and recommendation model | |
| CN114265986B (en) | Information pushing method and system fusing knowledge graph structure and path semantics | |
| CN113191530B (en) | Block link point reliability prediction method and system with privacy protection function | |
| CN113762595A (en) | Traffic time prediction model training method, traffic time prediction method and equipment | |
| CN110263236A (en) | Social network user multi-tag classification method based on dynamic multi-view learning model | |
| CN113487018A (en) | Global context enhancement graph neural network method based on session recommendation | |
| CN116431914A (en) | Cross-domain recommendation method and system based on personalized preference transfer model | |
| CN116166875A (en) | Bidirectional cross-domain recommendation method of heterogeneous graph neural network based on element path enhancement | |
| CN115470994A (en) | Information popularity prediction method and system based on explicit time and cascade attention | |
| CN114372680A (en) | Spatial crowdsourcing task allocation method based on worker loss prediction | |
| CN112925953B (en) | Dynamic network representation method and system | |
| Wang | A neural network method of density estimation for univariate unimodal data | |
| Zhu | A novel social network measurement and perception pattern based on a multi-agent and convolutional neural network | |
| WO2022186808A1 (en) | Method for solving virtual network embedding problem in 5g and beyond networks with deep information maximization using multiple physical network structure | |
| Danesh et al. | Predicting trust in a social network based on structural similarities using a multi-layered perceptron neural network | |
| CN112882914A (en) | Multi-dimensional measurement system for health of open-source software ecosystem | |
| Xi et al. | Intuitionistic fuzzy petri nets model based on back propagation algorithm for information services | |
| Jeon et al. | Scalable private p2p network for distributed and hierarchical machine learning in vanets | |
| CN117274616B (en) | Multi-feature fusion deep learning service QoS prediction system and prediction method | |
| Cui et al. | Large-scale network decomposition and mathematical programming based web service composition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |