CN113609306A - Social network link prediction method and system for resisting residual image variation self-encoder - Google Patents
Social network link prediction method and system for resisting residual image variation self-encoder Download PDFInfo
- Publication number
- CN113609306A CN113609306A CN202110893417.5A CN202110893417A CN113609306A CN 113609306 A CN113609306 A CN 113609306A CN 202110893417 A CN202110893417 A CN 202110893417A CN 113609306 A CN113609306 A CN 113609306A
- Authority
- CN
- China
- Prior art keywords
- graph
- social network
- data
- encoder
- users
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 239000013598 vector Substances 0.000 claims abstract description 43
- 230000003993 interaction Effects 0.000 claims abstract description 10
- 238000005516 engineering process Methods 0.000 claims abstract description 8
- 230000002452 interceptive effect Effects 0.000 claims abstract description 6
- 238000009826 distribution Methods 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 15
- 230000003042 antagnostic effect Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 6
- 238000000547 structure data Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000006855 networking Effects 0.000 claims 1
- 238000002474 experimental method Methods 0.000 description 11
- 241000689227 Cora <basidiomycete fungus> Species 0.000 description 8
- 238000012549 training Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000006872 improvement Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000008034 disappearance Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 102000004169 proteins and genes Human genes 0.000 description 2
- 108090000623 proteins and genes Proteins 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 241000370685 Arge Species 0.000 description 1
- 101000588749 Homo sapiens N-acetylglutamate synthase, mitochondrial Proteins 0.000 description 1
- 102100032618 N-acetylglutamate synthase, mitochondrial Human genes 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 206010012601 diabetes mellitus Diseases 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003012 network analysis Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000007430 reference method Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/18—Network design, e.g. design based on topological or interconnect aspects of utility systems, piping, heating ventilation air conditioning [HVAC] or cabling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/02—CAD in a network environment, e.g. collaborative CAD or distributed simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Biomedical Technology (AREA)
- Computational Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a social network link prediction method and a social network link prediction system for a residual error resisting graph variation self-encoder, which comprise the following steps: acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises tweet data of users and interactive data among the users; performing social network graph structure abstraction based on user interaction, wherein nodes represent real users in a social network, and edges represent relationships between the users; extracting content semantic information in the user tweet data by using a Bert model, and expressing the content semantic information into a vector with a fixed length as the content semantic of the user; taking a social network graph structure and content semantics of a user as input, extracting topological structure features and semantic features by using a confrontation residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space; and calculating dot products between every two node vector representations, reflecting the similarity between the nodes, and identifying the two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
Description
Technical Field
The invention belongs to the technical field of neural networks, and particularly relates to a social network link prediction method and system based on a residual image confrontation graph variation self-encoder under batch regularization.
Background
Link Prediction (Link Prediction) is one of the applications of Knowledge-Graph Embedding (Knowledge Graph Embedding), which maps the contents of entities and relations in a Knowledge-Graph into a continuous vector space, and predicts the entities or relations in the Knowledge-Graph, i.e., (h, r,. Based on the link prediction of the graph structure, the more popular methods include algorithms based on similarity, probability statistics, preprocessing, SVM or KNN and the like. Link prediction also includes representation learning based reasoning, neural network based reasoning, rule based reasoning, and hybrid reasoning.
Link prediction can be applied in a number of areas. The current application is more extensive: 1) acquaintances and similar users are recommended to users in social networks, and most social networks use link prediction techniques to recommend acquaintances. 2) In the biological field, link prediction is used to find proteins that can interact. Because many proteins are not familiar to people at present, and the time and money cost of the experiment is high, more accurate prediction is needed, and the cost is reduced. 3) For predicting the type of untagged node in a network of known partial node types, e.g. for determining the type of an academic paper or for predicting certain criminal activities from a criminal network.
In recent years, it has been shown that learning techniques have achieved remarkable results in the fields of natural language processing and the like, and research and application of graph embedding techniques have been inspired and promoted. The core goal of graph embedding is to map network topology information, node attribute information and edge attribute information into a low-dimensional, dense and continuous feature space, so that efficient graph data analysis, such as social network analysis, recommendation systems, epidemic propagation analysis, molecular property prediction, etc., can be performed by using the existing model. In order to achieve the aim, researchers have purposefully proposed a graph embedding algorithm based on a probability model, a graph embedding algorithm based on matrix decomposition and a graph embedding algorithm based on deep learning from different angles. The graph embedding is carried out based on deep learning, huge potential and advantages are shown, and various indexes are refreshed in tasks such as link prediction and node clustering.
Disclosure of Invention
The purpose of the invention is realized by the following technical scheme.
In order to realize more effective graph embedding, the invention provides a new graph variation self-encoder framework, which utilizes Batch regularization (Batch Normalization) to adjust approximate posterior parameters, so that KL follows the distribution of the whole data set, the expectation of KL distribution is ensured to be positive, and posterior collapse is avoided; in addition, by introducing the residual connecting and resisting module, the topological information and the content information of the graph can be embedded into the vector representation more stably, and the expression capability of the latent variable is enhanced. Results of link prediction experiments on three citation data sets show that the AUC score of the model provided by the invention is higher than 92%, the average precision is higher than 93%, and the model is competitive with the current best graph variation self-encoder.
According to a first aspect of the present invention, there is provided a social network link prediction method against a residual graph variation self-encoder, comprising the steps of:
acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises tweet data of users and interactive data among the users;
performing social network graph structure abstraction based on user interaction, wherein nodes represent real users in a social network, and edges represent relationships between the users;
extracting content semantic information in the user tweet data by using a Bert model, and expressing the content semantic information into a vector with a fixed length as the content semantic of the user;
taking a social network graph structure and content semantics of a user as input, extracting topological structure features and semantic features by using a confrontation residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space;
and calculating dot products between every two node vector representations, reflecting the similarity between the nodes, and identifying the two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
Further, the robust residual graph variation self-encoder under the batch regularization comprises: the graph self-encoder module and the confrontation network module.
Further, the user-based interaction performs social network diagram structure abstraction, including:
using GCN to extract the features of the non-Euclidean graph structure data, and expressing the propagation rule of each layer of graph convolution network as follows:
whereinIs the degree matrix of the graph G,is formed by adding a self-looping adjacency matrix, W(l)Andis the weight matrix and activation function of the current graph convolution layer; the graph is defined as G ═ { V, E, X }, where V ═ V }1,v2,...,vnDenotes a set of all nodes in the graph structure data, and E denotes a connection node v in the graphiAnd vjEdge e ofijSet of (2), xiE X represents a node viThe content characteristics of (a); z is a latent variable and is the embedding matrix of graph G.
Further, the robust residual graph variation self-encoder under the batch regularization comprises: and the residual error network is used for superposing the original input information of the graph volume layer on the basis of the local feature abstraction of the graph volume layer.
Further, in the antagonistic residual image variation self-encoder under the batch regularization, the distribution of the mean vector and the variance vector is fixed through the batch regularization operation, so that a positive lower bound is ensured to be expected for the KL divergence term.
Further, the distribution of the latent variables is determined by using the mean vector and the variance vector obtained after batch regularization, and then the latent variables are obtained through random sampling.
Further, the antagonistic residual image variation self-encoder under batch regularization further comprises a discriminator used for forcing latent variables to match prior distribution, taking Gaussian distribution as prior distribution and taking a standard multilayer perceptron as the discriminator.
According to a second aspect of the present invention, there is provided a social network link prediction system against residual graph variation autoencoder, comprising:
the data acquisition module is used for acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises the tweet data of users and the interactive data among the users;
the network graph structure abstraction module is used for abstracting the structure of the social network graph based on the interaction of users, wherein the nodes represent real users in the social network, and the edges represent the relationship among the users;
the semantic extraction module is used for extracting content semantic information in the user tweet data by using a Bert model, expressing the content semantic information into a vector with a fixed length, and using the vector as the content semantic of the user;
the node representation module is used for taking the social network graph structure and the content semantics of the user as input, extracting topological structure characteristics and semantic characteristics by using an antagonistic residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space;
and the link prediction module is used for calculating dot products between the node vector representations pairwise, reflecting the similarity between the nodes, and identifying two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
The advantages of the invention are summarized as follows:
1. a new graph embedding framework is provided, and the network topology structure and the node characteristics can be effectively mapped to a continuous vector space, so that better node representation is provided for a graph analysis task.
2. By using batch regularization and residual connection, the stability of the graph variation self-encoder is improved, and the problems of KL divergence disappearance, gradient dispersion and the like are avoided.
3. Experiments on the proposed model were performed on a reference dataset, and the results demonstrate the superiority of the model over other graph variation autoencoders.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a flow chart of a social network link prediction method based on a confrontation residual graph variation self-encoder under batch regularization according to an embodiment of the invention.
Fig. 2 shows an overall architecture diagram of a robust residual map variation self-encoder (BNAVGE) under batch regularization according to an embodiment of the present invention.
Fig. 3 shows a graph variable component encoder applied to a Cora data set to obtain corresponding graph embedding, and a graph variable component encoder applied to a Cora data set to perform dimension reduction and visualize the result by using a t-SNE algorithm according to an embodiment of the present invention.
Fig. 4 shows a block diagram of a social network link prediction system based on a confrontational residual graph variation self-encoder under batch regularization according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 6 shows a schematic diagram of a storage medium provided in an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
According to an embodiment of the invention, a social network link prediction method of an antagonistic residual graph variation self-encoder based on batch regularization is provided, which comprises the following steps:
step 101: acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises tweet data of users and interactive data among the users; the data collection can use web crawlers and other technologies. The user's tweet data may include, but is not limited to, common forms of network media such as WeChat, microblog, self-media, and the like.
Step 102: performing social network graph structure abstraction based on user interaction, wherein nodes represent real users in a social network, and edges represent relationships such as attention among the users;
step 103: semantic information in a user tweet is extracted by using a Bert model and expressed into a vector with a fixed length as the content semantic of the user;
step 104: taking a social network graph structure and content semantics of a user as input, extracting topological structure features and semantic features by using a confrontation residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space;
step 105: and calculating dot products between every two node vector representations, reflecting the similarity between the nodes, and identifying the two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
The graph variational self-encoder can carry out graph embedding under the unsupervised condition, and generates the feature representation of the nodes in a low-dimensional space. However, in the optimization process of the correlation model of the graph variable-component encoder, model degradation may be caused by disappearance of KL divergence, and the model converges to local optimum, so that the correlation model cannot be used for graph analysis tasks such as subsequent link prediction.
Aiming at the problem that the KL divergence is possibly weakened or disappeared, the invention introduces batch regularization operation and adjusts the output of the graph variable component encoder, so that the KL divergence has a lower bound larger than zero, thereby ensuring that the problem of posterior collapse cannot occur. In addition, in order to make the forward and backward propagation of information smoother in the model training process, residual error connection is added in the graph self-encoder, so that information loss caused by a graph convolution layer is made up; and the robustness and stability of the graph variation self-encoder are improved to a certain extent by utilizing the anti-network regularization latent variable. The experimental result on the reference data set shows that the algorithm provided by the invention can effectively embed the graph and has better performance on the unsupervised graph analysis task of link prediction.
The graph variation self-encoder and the related variants thereof have been proved to be capable of effectively embedding graphs, however, research on the problem of KL divergence disappearance of the graph variation self-encoder is relatively few, and the invention utilizes batch regularization to adjust the distribution of approximate posterior parameters on the basis of the existing research, thereby setting the expected lower bound of KL distribution and avoiding the potential problem of posterior collapse. In order to further improve the representation capability of the encoder, the method uses the residual error connection optimization graph convolution neural network, and meanwhile, uses the countermeasure network to force latent variables to match prior distribution, so as to ensure that the finally obtained graph embedding has robustness. Fig. 2 shows the overall architecture of the robust residual graph variation self-encoder (BNAVGE) under batch regularization, mainly including the graph self-encoder module and the robust network module, and compared with the related work of the existing graph embedding, the improvement of the BNAVGE model mainly comes from the optimization for the encoder part.
The graph in the present invention is defined as G ═ { V, E, X }, where V ═ V1,v2,...,vnDenotes a set of all nodes in the graph structure data, and E denotes a connection node v in the graphiAnd vjEdge e ofijSet of (2), xiE X represents a node viThe content characteristics of (1). To facilitate the labeling of whether there is an associated edge between two nodes, an adjacency matrix A is usedn×nRecording the topology structure information of graph G if node viAnd vjWith an edge in between, Aij1, otherwise Aij0. For node viOur goal is to project it into a low-dimensional continuous vector space, thereby enabling the use of vectorsRepresenting the node, in order to ensure that graph embedding can be used for graph analysis tasks, the vector representation of the node should retain as much more important topological information and semantic information as possible, namely:this is exactly what the encoder part needs to do, the latent variable Z ∈ Rn×dIs the embedded matrix of fig. G.
The core task of the encoder is to fuse the topological structure characteristics of the network and the content characteristics of the nodes to obtain the embedded representation of the nodes. The invention selects to use graph neural network (GCN) to extract the features of non-Euclidean graph structure data, and the propagation rule of each layer of graph convolutional network can be expressed as:
whereinIs the degree matrix of the graph G,is formed by adding a self-looping adjacency matrix, W(l)Andis the weight matrix and activation function of the layer 1 map convolutional layer. In particular, we have Z(0)X to ensure that the model can exploit the content features to the nodes.
In order to make the model easier to optimize and make the further increase of the depth of the graph neural network possible, the invention introduces a residual error network in the encoder part, and superimposes the original input information of the graph convolution layer on the basis of the local feature abstraction of the graph convolution layer, and the propagation rule is as follows:
and the graph variation autoencoder is defined by an inference model, namely:
q is the distribution of the latent variable Z, N represents the normal distribution, and the mean vector muiAnd variance vector sigmaiThe corresponding matrices can be fitted by different GCNs:
μ=GCNμ(X,A)
logσ=GCNσ(X,A)
the problem that KL divergence disappears possibly occurs in the training process of the variational self-encoder, and finally a decoder does not use the information of the latent variable any more, a model is degraded, and the situation that the latent variable exists cannot be obtainedAn embedded representation of the effect. In the graph variation auto-encoder, we can operate on μ by Batch regularization (BN)iAnd σiIs fixed to ensure that there is a positive lower bound on the expectation of the KL divergence term, avoiding posterior collapse. Batch regularization can be mathematically expressed as:
γ and β represent the scaling factor and offset, respectively, at batch regularization, where μxAnd σxThe mean and standard deviation of x are indicated, respectively.
In particular, we immobilize βμ=βσTwo scaling coefficient vectors are introduced simultaneously, 0:
where τ ∈ (0, 1) is a constant and θ is a trainable parameter. Then we will adjust μ and σ as follows:
where γ is the corresponding scaling factor, μμAnd sigmaμMean and standard deviation of the variable μ, respectively; u. ofбAnd sigmaбMean and standard deviation, respectively, of the variable Be.
Obtained by batch regularizationAndwe can determine the distribution of the latent variables and then obtain the latent variables by random sampling.
As for the decoder part, we try to reconstruct the topology of graph G, using the inner product of two node latent vectors to predict whether there is an edge between the two nodes i, j (N is the number of nodes):
where p is the corresponding distribution, zi TIs the transpose of the latent variable of the node. Thus, the reconstructed topologyCan be expressed as:
in the training process of the variational self-encoder, the optimization goal is usually to maximize the lower confidence bound (ELBO), which is equivalent to maximizing the reconstruction term and minimizing the KL divergence term. If the latent variables can be well represented by nodes, the reconstructed topology structure of the decoder is very similar to the original topology structure, and the difference between the reconstructed topology structure and the original topology structure can be measured by reconstruction errors; furthermore, we also require that the posterior distribution q (Z | X, a) be as close as possible to the prior distribution p (Z), which we assume in the present invention is gaussian. Finally we can get the loss function of the graph variation self-encoder:
wherein L isreconAnd LKLThe reconstruction error and the KL divergence value, respectively, E represents the mathematical expectation.
The image self-encoder only restricts each dimension of the latent variable in the reasoning process, and ignores the data distribution of the latent variable, which may cause that the embedding effect in the real image data is not ideal, so that the algorithm thought of generating the countermeasure network can be consulted, and a discriminator is added on the basis of the image variation self-encoder to force the latent variable to match the prior distribution. The invention takes Gaussian distribution as prior distribution, takes a standard multilayer perceptron (MLP) as a discriminator, and the training target of the discriminator is to distinguish whether input is from prior distribution or latent variable, so the discriminator is actually a two-classifier, and usually uses cross entropy as a loss function:
where E represents the mathematical expectation, D represents the corresponding probability distribution, and g (the letter g for the flower in the above formula) represents the generative model (the encoder portion of the graph self-encoder).
The graph self-encoder, or more precisely the encoder part, is jointly optimized with a discriminator which distinguishes as much as possible whether the samples come from a prior distribution or a latent variable, while the encoder tries to confuse the two, resulting in a more robust graph-embedded representation:
in order to accurately evaluate the performance of the BNAVGE model proposed by the present invention, we will perform graph embedding on three reference data sets, and use the embedded representation of nodes to complete the unsupervised graph analysis task of link prediction, and compare it with the existing excellent-of-the-art model.
A. Data set
In graph representation learning, three citation data sets of Cora, Citeser and Pubmed are most commonly used, wherein Cora and Citeser are networks formed by mutual citation among computer science-related publications, Pubmed is a citation network formed by a group of scientific publications related to diabetes, nodes represent publications, edges represent citation relations, characteristics are unique heat codes or TF-IDF code vectors of the nodes under corresponding dictionaries, and labels are categories to which the nodes belong. The statistics of the data set are shown in table 1.
Table 1.Datasets
Cora | Citeseer | Pubmed | |
Nodes | 2708 | 3327 | 19717 |
Edges | 5429 | 4732 | 44338 |
Feature | 1433 | 3703 | 500 |
Labels | 7 | 6 | 3 |
B. Reference line
To demonstrate the performance advantages of the BNAVGE model proposed by the present invention, we compared it with six top-ranked graph embedding algorithms:
a) spectral Clustering: the graph theory-based clustering method divides a weighted undirected graph into two or more optimal subgraphs, and has the basic idea that clustering is performed by using a feature vector obtained after performing feature decomposition on a Laplacian matrix of sample data, so that a sample space in any shape can be identified and converged to a global optimal solution.
b) Deep Walk: the node local structure information can be captured based on the submerged representation of the maximum node co-occurrence, and the expansibility is strong.
c) GAE and VGAE: the graph self-encoder proposed by Thomas N.Kipf is used for graph embedding for the first time, and the graph embedding performance is greatly leaped by virtue of a simple encoder-decoder structure and high-efficiency encoder capability.
d) ARGA and ARVGA: on the basis of GAE and VGAE, the graph embedding model which is most stable and optimal in performance at present is used for minimizing reconstruction errors and simultaneously utilizing countertraining regularization latent variables.
For the above reference method, we set up according to the corresponding parameters.
C. Procedure of experiment
In the experiment, we built a neural network model using Tensorflow1.4 and performed model training on NVIDIA 1050 ti. The experimental setup is different considering the obvious difference in the complexity of the networks in the three reference datasets. For the Cora and cineseer datasets, the learning rates of the graph autoencoder and discriminator were set to 0.002 and 0.001, respectively, and the model was optimized 200 iterations using Adam algorithm. And the Pubmed data set is large, in order to ensure that the model can be trained fully, the training round is increased to 2000 rounds, and the learning rates of the graph self-encoder and the discriminator are set to be 0.005. We used AUC score and average Accuracy (AP) as evaluation criteria for link prediction effect, and we performed 10 times per group of experiments due to the randomness of the experiments, and recorded the average and standard error as the final result. Each data set is divided into a training set, a verification set and a test set, wherein 5% of edges in the verification set are used for hyper-parameter optimization, and 10% of edges in the test set are used for performance evaluation.
D. Results of the experiment
Table 2.Result
The results of the link prediction experiments are shown in table 2. Compared with other graph embedding models, the BNAVGE model provided by the invention achieves the best results in link prediction experiments on Cora and Citeser data sets, the AUC score is higher than 92%, and the AP score is higher than 93%; there is also a very small gap (about 0.3%) in the Pubmed data set compared to the current best performing model (ARGE). If the correlation model (VGAE) of the graph-based variable component encoder is compared purely*VGAE, ARVGE and BNAVGE), BNAVGE has the best performance on each data set, with an improvement of AUC score of at least 1.1% and an improvement of AP of at least 0.7% compared to VGAE. In addition, we modify the VGAE model (VGAE + Res + BN), add the residual network and batch regularization operation, and also bring the performance improvement, which shows that the improvement we do can optimize the graph variation self-encoder and enhance the effect of embedded representation.
E. Visualization
The graph variable component encoder provided by the invention is applied to a Cora data set to obtain corresponding graph embedding, and dimension reduction and visualization are carried out by utilizing a t-SNE algorithm, so that as a result is shown in FIG. 3, the node representations of the same type (same color) have a more obvious aggregation phenomenon, and the graph embedding effectiveness is reflected from the side.
The invention provides a new graph variation self-encoder framework BNAVGE, which solves the problem of potential KL divergence disappearance by batch regularization under the condition of not obviously increasing the training difficulty, and simultaneously introduces a residual error network and a countermeasure network to further optimize the generation effect of an encoder on node embedding representation. Experiments show that the algorithm can generate robust representation of a low-dimensional continuous space based on the topological structure of the graph and the content semantics of the nodes, and obtains a result superior to a baseline in a link prediction task. In fact, the encoder and decoder structure of the BNAVGE model that we designed is still relatively simple, considering that we can avoid the model degradation problem caused by the disappearance of KL divergence and the increase of the number of network layers in the improvement of the graph variation self-encoder, we will try to use a more complex graph neural network for feature extraction in the future, and simultaneously use a generating network to replace an inner product decoder, thereby seeking a graph embedding representation with stronger expressive power.
The application embodiment provides a social network link prediction apparatus for an anti-residual graph variation self-encoder, which is configured to perform the social network link prediction method for the anti-residual graph variation self-encoder according to the foregoing embodiment, as shown in fig. 4, the apparatus includes:
the data acquisition module 501 is configured to acquire social network data at a certain time by using a data acquisition technology, where the social network data includes tweet data of users and interaction data between the users;
a network graph structure abstraction module 502 for abstracting the structure of the social network graph based on the interaction of the users, wherein the nodes represent real users in the social network and the edges represent the relationship between the users;
the semantic extraction module 503 is configured to extract content semantic information in the user tweet data by using a Bert model, and express the content semantic information as a vector with a fixed length as the content semantic of the user;
a node representation module 504, configured to take a social network graph structure and content semantics of a user as inputs, extract topological structure features and semantic features by using an antagonistic residual graph variation self-encoder under batch regularization, and obtain node representation in a low-dimensional continuous vector space through fusion;
and the link prediction module 505 is used for calculating dot products between the node vector representations pairwise, reflecting the similarity between the nodes, and identifying two nodes higher than a given threshold value as being capable of generating a link relation in the future so as to realize the link prediction of the social network.
The social network link prediction device of the anti-residual image variation self-encoder provided by the above embodiment of the application and the social network link prediction method of the anti-residual image variation self-encoder provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the application program stored in the device.
The embodiment of the present application further provides an electronic device corresponding to the social network link prediction method of the confrontation residual image variation self-encoder provided in the foregoing embodiment, so as to execute the social network link prediction method of the confrontation residual image variation self-encoder. The embodiments of the present application are not limited.
Please refer to fig. 5, which illustrates a schematic diagram of an electronic device according to some embodiments of the present application. As shown in fig. 5, the electronic device 2 includes: the system comprises a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; the memory 201 stores a computer program that can be executed on the processor 200, and the processor 200 executes the computer program to execute the method for predicting social network links against residual graph variation autoencoder provided in any of the foregoing embodiments of the present application.
The Memory 201 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the apparatus and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
The processor 200 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 200. The Processor 200 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the method in combination with the hardware thereof.
The electronic device provided by the embodiment of the application and the social network link prediction method for the antagonistic residual image variation self-encoder provided by the embodiment of the application have the same beneficial effects as the method adopted, operated or realized by the electronic device.
Referring to fig. 6, the computer readable storage medium is an optical disc 30, on which a computer program (i.e., a program product) is stored, and when the computer program is executed by a processor, the computer program performs the social network link prediction method of the robust residual image variation self-encoder according to any of the foregoing embodiments.
It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, or other optical and magnetic storage media, which are not described in detail herein.
The computer-readable storage medium provided by the above-mentioned embodiment of the present application and the social network link prediction method against the residual graph variation self-encoder provided by the embodiment of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored in the computer-readable storage medium.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the creation apparatus of a virtual machine according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present application, and these should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A social network link prediction method for an anti-residual graph variation self-encoder is characterized by comprising the following steps:
acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises tweet data of users and interactive data among the users;
performing social network graph structure abstraction based on user interaction, wherein nodes represent real users in a social network, and edges represent relationships between the users;
extracting content semantic information in the user tweet data by using a Bert model, and expressing the content semantic information into a vector with a fixed length as the content semantic of the user;
taking a social network graph structure and content semantics of a user as input, extracting topological structure features and semantic features by using a confrontation residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space;
and calculating dot products between every two node vector representations, reflecting the similarity between the nodes, and identifying the two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
2. A method according to claim 1, characterized in that:
the antagonistic residual image variation self-encoder under batch regularization comprises: the graph self-encoder module and the confrontation network module.
3. A method according to claim 1 or 2, characterized in that:
the user-based interaction for social network graph structure abstraction comprises:
using GCN to extract the features of the non-Euclidean graph structure data, and expressing the propagation rule of each layer of graph convolution network as follows:
whereinIs the degree matrix of the graph G,is formed by adding a self-looping adjacency matrix, W(l)Andis the weight matrix and activation function of the current graph convolution layer; the graph is defined as G ═ { V, E, X }, where V ═ V }1,v2,...,vnDenotes a set of all nodes in the graph structure data, and E denotes a connection node v in the graphiAnd vjEdge e ofijSet of (2), xiE X represents a node viThe content characteristics of (a); z is a latent variable and is the embedding matrix of graph G.
4. A method according to claim 3, characterized in that:
the antagonistic residual image variation self-encoder under batch regularization comprises: and the residual error network is used for superposing the original input information of the graph volume layer on the basis of the local feature abstraction of the graph volume layer.
5. A method according to claim 4, characterized in that:
in the confrontation residual image graph variable self-encoder under the batch regularization, the distribution of a mean vector and a variance vector is fixed through the batch regularization operation, and a positive lower bound of the expectation of the KL divergence term is ensured.
6. A method according to claim 5, characterized by:
and determining the distribution of the latent variables by using the mean vector and the variance vector obtained after batch regularization, and obtaining the latent variables by random sampling.
7. A method according to claim 6, characterized by:
the confrontation residual image variation self-encoder under batch regularization further comprises a discriminator used for forcing latent variables to be matched with prior distribution, taking Gaussian distribution as prior distribution and taking a standard multilayer perceptron as the discriminator.
8. A social networking link prediction system against residual graph variation autocoder, comprising:
the data acquisition module is used for acquiring social network data at a certain moment by using a data acquisition technology, wherein the social network data comprises the tweet data of users and the interactive data among the users;
the network graph structure abstraction module is used for abstracting the structure of the social network graph based on the interaction of users, wherein the nodes represent real users in the social network, and the edges represent the relationship among the users;
the semantic extraction module is used for extracting content semantic information in the user tweet data by using a Bert model, expressing the content semantic information into a vector with a fixed length, and using the vector as the content semantic of the user;
the node representation module is used for taking the social network graph structure and the content semantics of the user as input, extracting topological structure characteristics and semantic characteristics by using an antagonistic residual graph variation self-encoder under batch regularization, and fusing to obtain node representation in a low-dimensional continuous vector space;
and the link prediction module is used for calculating dot products between the node vector representations pairwise, reflecting the similarity between the nodes, and identifying two nodes higher than a given threshold value as generating a link relation in the future so as to realize the link prediction of the social network.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor to implement the method according to any of claims 1-7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110893417.5A CN113609306B (en) | 2021-08-04 | 2021-08-04 | Social network link prediction method and system for anti-residual diagram variation self-encoder |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110893417.5A CN113609306B (en) | 2021-08-04 | 2021-08-04 | Social network link prediction method and system for anti-residual diagram variation self-encoder |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113609306A true CN113609306A (en) | 2021-11-05 |
CN113609306B CN113609306B (en) | 2024-04-23 |
Family
ID=78339552
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110893417.5A Active CN113609306B (en) | 2021-08-04 | 2021-08-04 | Social network link prediction method and system for anti-residual diagram variation self-encoder |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113609306B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114912033A (en) * | 2022-05-16 | 2022-08-16 | 重庆大学 | Knowledge graph-based recommendation popularity deviation adaptive buffering method |
CN116975311A (en) * | 2023-09-15 | 2023-10-31 | 江西农业大学 | Agricultural pest knowledge graph optimization method, system and computer |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190130212A1 (en) * | 2017-10-30 | 2019-05-02 | Nec Laboratories America, Inc. | Deep Network Embedding with Adversarial Regularization |
CN111414478A (en) * | 2020-03-13 | 2020-07-14 | 北京科技大学 | Social network emotion modeling method based on deep cycle neural network |
CN111767472A (en) * | 2020-07-08 | 2020-10-13 | 吉林大学 | Method and system for detecting abnormal account of social network |
CN111784081A (en) * | 2020-07-30 | 2020-10-16 | 南昌航空大学 | Social network link prediction method adopting knowledge graph embedding and time convolution network |
CN112417219A (en) * | 2020-11-16 | 2021-02-26 | 吉林大学 | Hyper-graph convolution-based hyper-edge link prediction method |
CN113052712A (en) * | 2021-03-05 | 2021-06-29 | 浙江师范大学 | Social data analysis method and system and storage medium |
-
2021
- 2021-08-04 CN CN202110893417.5A patent/CN113609306B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190130212A1 (en) * | 2017-10-30 | 2019-05-02 | Nec Laboratories America, Inc. | Deep Network Embedding with Adversarial Regularization |
CN111414478A (en) * | 2020-03-13 | 2020-07-14 | 北京科技大学 | Social network emotion modeling method based on deep cycle neural network |
CN111767472A (en) * | 2020-07-08 | 2020-10-13 | 吉林大学 | Method and system for detecting abnormal account of social network |
CN111784081A (en) * | 2020-07-30 | 2020-10-16 | 南昌航空大学 | Social network link prediction method adopting knowledge graph embedding and time convolution network |
CN112417219A (en) * | 2020-11-16 | 2021-02-26 | 吉林大学 | Hyper-graph convolution-based hyper-edge link prediction method |
CN113052712A (en) * | 2021-03-05 | 2021-06-29 | 浙江师范大学 | Social data analysis method and system and storage medium |
Non-Patent Citations (1)
Title |
---|
徐佳宇;张冬明;靳国庆;包秀国;袁庆升;张勇东;: "PNET:像素级台标识别网络", 计算机辅助设计与图形学学报, no. 10, 15 October 2018 (2018-10-15) * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114912033A (en) * | 2022-05-16 | 2022-08-16 | 重庆大学 | Knowledge graph-based recommendation popularity deviation adaptive buffering method |
CN116975311A (en) * | 2023-09-15 | 2023-10-31 | 江西农业大学 | Agricultural pest knowledge graph optimization method, system and computer |
CN116975311B (en) * | 2023-09-15 | 2023-12-01 | 江西农业大学 | Agricultural pest knowledge graph optimization method, system and computer |
Also Published As
Publication number | Publication date |
---|---|
CN113609306B (en) | 2024-04-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Cai et al. | A comprehensive survey of graph embedding: Problems, techniques, and applications | |
Trung et al. | A comparative study on network alignment techniques | |
Meel et al. | A temporal ensembling based semi-supervised ConvNet for the detection of fake news articles | |
Zhu et al. | A survey on graph structure learning: Progress and opportunities | |
Hui et al. | Collaborative graph convolutional networks: Unsupervised learning meets semi-supervised learning | |
Waikhom et al. | A survey of graph neural networks in various learning paradigms: methods, applications, and challenges | |
Sun et al. | Dual-decoder graph autoencoder for unsupervised graph representation learning | |
Lim et al. | Efficient-prototypicalnet with self knowledge distillation for few-shot learning | |
CN112417289A (en) | Information intelligent recommendation method based on deep clustering | |
CN113609306B (en) | Social network link prediction method and system for anti-residual diagram variation self-encoder | |
Chen et al. | An ensemble model for link prediction based on graph embedding | |
Liu et al. | Weakly supervised image classification and pointwise localization with graph convolutional networks | |
CN110737730A (en) | Unsupervised learning-based user classification method, unsupervised learning-based user classification device, unsupervised learning-based user classification equipment and storage medium | |
Li et al. | Multi-view representation model based on graph autoencoder | |
Mesgaran et al. | Anisotropic graph convolutional network for semi-supervised learning | |
Fu et al. | hier2vec: interpretable multi-granular representation learning for hierarchy in social networks | |
Li et al. | CSAT: Contrastive Sampling-Aggregating Transformer for Community Detection in Attribute-Missing Networks | |
Arya et al. | Node classification using deep learning in social networks | |
Guo et al. | DP-DDCL: A discriminative prototype with dual decoupled contrast learning method for few-shot object detection | |
Shi et al. | Advances in Graph Neural Networks | |
Wang et al. | An Improved Convolutional Neural Network‐Based Scene Image Recognition Method | |
Rathee et al. | A machine learning approach to predict the next word in a statement | |
CN114332469A (en) | Model training method, device, equipment and storage medium | |
Zhang et al. | A multi-view mask contrastive learning graph convolutional neural network for age estimation | |
Liu et al. | Network representation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |