CN117610717A - Information popularity prediction method based on double-variation cascade self-encoder - Google Patents
Information popularity prediction method based on double-variation cascade self-encoder Download PDFInfo
- Publication number
- CN117610717A CN117610717A CN202311502493.4A CN202311502493A CN117610717A CN 117610717 A CN117610717 A CN 117610717A CN 202311502493 A CN202311502493 A CN 202311502493A CN 117610717 A CN117610717 A CN 117610717A
- Authority
- CN
- China
- Prior art keywords
- cascade
- information
- graph
- encoder
- self
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000009792 diffusion process Methods 0.000 claims abstract description 56
- 238000010586 diagram Methods 0.000 claims abstract description 42
- 230000003993 interaction Effects 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims description 37
- 239000013598 vector Substances 0.000 claims description 30
- 238000009826 distribution Methods 0.000 claims description 26
- 230000006870 function Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 18
- 230000008859 change Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 10
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000013528 artificial neural network Methods 0.000 abstract 1
- 238000011176 pooling Methods 0.000 description 9
- 238000000354 decomposition reaction Methods 0.000 description 5
- 238000011160 research Methods 0.000 description 4
- 101100481876 Danio rerio pbk gene Proteins 0.000 description 2
- 101100481878 Mus musculus Pbk gene Proteins 0.000 description 2
- 238000002679 ablation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 206010044565 Tremor Diseases 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 125000003275 alpha amino acid group Chemical group 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/01—Social networking
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Primary Health Care (AREA)
- Biophysics (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Game Theory and Decision Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides an information popularity prediction method based on a double-variation cascade self-encoder, which comprises the steps of obtaining a global interaction diagram and a cascade diagram in an information diffusion process; an information diffusion model is built, a global interaction diagram and a cascade diagram are input into the information diffusion model, wherein a basic information diffusion model comprises a variation diagram self-encoder and a variation time sequence self-encoder; and outputting a popularity prediction result of the information through the information diffusion model. According to the method provided by the invention, the propagation topological structure and the reasoning propagation process are fitted based on the graph neural network technology, key factors influencing information propagation are captured, and a unified information diffusion prediction framework is constructed, so that a more accurate prediction result is obtained on an information popularity prediction task.
Description
Technical Field
The invention belongs to the technical field of network space security.
Background
With the full popularization of smart phones, digital information generated by social media platforms such as microblogs, weChats and even tremble sounds every day is exponentially expanded, the platforms greatly promote the generation and propagation of information, so that the problems of information overload, false information flooding and the like are caused, meanwhile, the competition of different information on the attention of users is also aggravated, and information diffusion prediction provides possibility for solving the problems. Information diffusion prediction (Information Diffusion Prediction) refers to capturing the pre-propagation process (also called cascading) of information and related data in a social platform, analyzing which factors affect information propagation, mining implicit rules therein, and predicting the propagation process or propagation result of the information in the future through modeling. Specific research directions include the Popularity of predicted information, the influence of quantized information, the target audience of inferred information and the like, and the research is widely applied to real social media mining. From the perspective of marketers, the heat of the forecast information can help the marketers to discover potential hot spot products in advance, and provide guidance for decisions of the marketers, so that related applications such as virus marketing, influence maximization, advertisement recommendation and the like are driven; from the perspective of a social platform manager, the popularity of the predicted information can help the manager to predict and limit the propagation of illegal or harmful information, thereby being used as an auxiliary basis for tasks such as false information detection and the like.
However, the problem of information popularity prediction faces many challenges, namely, firstly, the collected cascade data have deletion and deviation, and when the observation period is shorter, the user behavior data are less in the information cascade, secondly, the interactive behaviors such as forwarding of the user have uncertainty and complexity, thirdly, the information popularity is distributed in a heavy tail, and the extreme value sample increases the prediction difficulty. Therefore, the information diffusion prediction research needs to fully utilize the information cascade (Information Cascade), and the inherent association and combination modeling between the information cascade and the data such as the bottom social network, the global interaction diagram and the like are explored, and meanwhile, the cascade data has the double characteristics of a time sequence and a topological structure, so that the final prediction effect of the model is directly related to how to effectively mine and utilize the two characteristics.
The information popularity prediction model is applied to the actual situation that the calculation cost and the generalization capability of the model need to be considered with great importance. In view of cost, the method has higher difficulty in completely obtaining the underlying social network on which the information cascade depends, and higher calculation cost is required for processing the huge dense social network adjacency matrix, so the method takes the global interaction network generated by the information cascade collaborative processing as the underlying network, and sparsifies the dense matrix to reduce the calculation cost. For model generalization capability considerations, the overlap between the training cascade and the actual cascade of models is typically small. Compared with a discriminant model, the generated model meets the requirement of an incremental learning mode in practice. The AECasN and other studies all use a variational self-encoder and other generative models to model information diffusion. The present invention also contemplates the use of a variational self-encoder to generate implicit distributions of user feature vectors, based on which a new user's implicit variable representation can be derived, thereby enhancing the generalization ability of the model. Based on the above-mentioned idea, considering that the dual characteristics of the structure and the time sequence of the cascade graph are critical to the modeling information diffusion process, the optimization objective of the popularity prediction task is to predict the forwarding increment (i.e. calculate the mean square error between the true value and the predicted value) of a period of time in the future, and the evaluation and constraint on the structural feature and the time sequence variation feature are absent. The present invention contemplates the use of a variational self-encoder to model structural and temporal features, respectively, and jointly optimize the reconstruction loss of the encoder with the predictive mean square error of popularity. In addition, the global social graph contains the influence of the users, the local cascade graph contains the propagation structure of popular or non-popular information, and the global diffusion graph contains the historical interaction among the users, but the research on comprehensively utilizing the factors is less at present.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems in the related art to some extent.
Therefore, the invention aims to provide an information popularity prediction method based on a double-variation cascade self-encoder, which is used for information popularity prediction.
To achieve the above object, an embodiment of a first aspect of the present invention provides an information popularity prediction method based on a double-variation cascade self-encoder, including:
acquiring a global interaction graph and a cascade graph in the information diffusion process;
an information diffusion model is built, the global interaction graph and the cascade graph are input into the information diffusion model, and the information diffusion model comprises a variation graph self-encoder and a variation time sequence self-encoder;
and outputting a popularity prediction result of the information through the information diffusion model.
In addition, the information popularity prediction method based on the double-variation cascade self-encoder according to the embodiment of the present invention may further have the following additional technical features:
further, in an embodiment of the present invention, the inputting the global interaction graph and the cascade graph into the information diffusion model includes:
processing the global interaction graph and the cascade graph respectively to obtain a global user embedding and a local cascade graph embedding, and connecting the global user embedding and the local cascade graph embedding in series;
using the result of the serial connection of the global user embedding and the local cascade diagram embedding as input, generating a low-dimensional implicit representation of nodes in the cascade diagram by using the variogram self-encoder, and generating a hidden variable representation representing the whole cascade sequence by using the variogram self-encoder;
the low-dimensional implicit representation is connected with the hidden variable representation in series and then is input to a multi-layer perceptron to conduct popularity prediction of the information.
Further, in an embodiment of the present invention, processing the cascade graph to obtain a local cascade graph embedding includes:
laplacian matrix L of the cascade graph C ＝D C -A C As an algorithm input;
eigenvalue lambda decomposed from the laplace matrix 0 ＜λ 1 ≤...≤λ M Corresponding to the signal frequency in the signal processing: if the feature value is larger, the corresponding feature vector will followThe topological structure of the graph is changed to be sensitive, if the characteristic value is smaller, the change amplitude of the corresponding characteristic vector is smaller, and the heat core g is heated s (λ)＝e -sλ As a graph filter kernel and designing a thermal kernel from the low-pass modulation effect to generate a smoothed embedded representation vector ψ=ue -sΛ U T The method comprises the steps of carrying out a first treatment on the surface of the Wherein ψ is u，s I.e. the embedded vector corresponding to user u, whereinIs psi u，s An mth wavelet coefficient component of (2);
the embedded vector elements of each node of the cascade diagram are regarded as a probability distribution of random variables, and the distribution is then represented by an empirical characteristic function, and the characteristic function of the random variable X is defined as Wherein for a given node u and scale parameter s, the empirical feature function is defined as +.>Meaning column mean calculation +.>The real part Re (phi) of the node in the cascade diagram u (t)) and imaginary value Im (phi) u (t)) to obtain the embedding of the node u in the cascade diagram>
Further, in an embodiment of the present invention, the step of using the variogram self-encoder to generate a low-dimensional implicit representation of the nodes in the cascade map by using the result of concatenating the global user-embedding and the local cascade map-embedding as input includes:
the graph attention network GAT is adopted for coding, and the original adjacency matrix is reconstructed in an inner product mode.
Further, in an embodiment of the present invention, the generating, with the variability sequential self-encoder, a hidden variable representation representing the whole concatenated sequence includes:
generating each intermediate implicit embedding with a bi-directional GRU network;
generating Gaussian distribution by using a DNN network;
sampling on the gaussian distribution generates a hidden variable representation of the corresponding node.
Further, in an embodiment of the present invention, the constructing an information diffusion model includes:
optimizing the information diffusion model using the loss function is expressed as:
wherein lambda is 1 Lambda is the structural loss 2 Is a timing penalty.
To achieve the above object, an embodiment of a second aspect of the present invention provides an information popularity prediction apparatus based on a double-variation cascade self-encoder, including:
the acquisition module is used for acquiring a global interaction graph and a cascade graph in the information diffusion process;
the input module is used for constructing an information diffusion model, and inputting the global interaction graph and the cascade graph into the information diffusion model, wherein the information diffusion model comprises a variational graph self-encoder and a variational time sequence self-encoder;
an output module for outputting popularity prediction results of the information through the information diffusion model
To achieve the above object, an embodiment of the third aspect of the present invention provides a computer device, which is characterized by comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor implements an information popularity prediction method based on a double variation cascade self-encoder as described above when executing the computer program.
To achieve the above object, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements an information popularity prediction method based on a double-variation cascade self-encoder as described above.
The embodiment of the invention provides an information popularity prediction method based on a double-variation cascade self-encoder, which provides a double-variation cascade self-encoder (DVCAE), enhances the generalization and the robustness of a prediction model by utilizing the generating capacity of the variation self-encoder, models the cascaded graph structure and time sequence change in parallel by utilizing the variation self-encoder, and optimizes the prediction loss, the structure reconstruction loss and the time sequence reconstruction loss in a combined way. The model not only has good prediction performance, but also can explicitly evaluate the learning of the model on the cascade graph structure and the learning of the time sequence characteristics, and ensures that the input characteristics are effectively compressed. According to visualization, ablation and parameter sensitivity experimental analysis, the correlation between the structural hidden variable representation and the information popularity is high, and the inherent mode of information diffusion can be well learned through modeling the propagation diagram structure, so that the future information popularity can be accurately predicted.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a schematic flow chart of an information popularity prediction method based on a double-variation cascade self-encoder according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an example social network provided by an embodiment of the present invention;
FIG. 3 is a schematic illustration of an information cascade provided by an embodiment of the present invention;
FIG. 4 is a schematic diagram of a network structure of a multi-layer perceptron provided by an embodiment of the present invention;
fig. 5 is a flow chart of an information popularity prediction method based on a double-variation cascade self-encoder according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.
An information popularity prediction method based on a double-variation cascade self-encoder according to an embodiment of the present invention is described below with reference to the accompanying drawings.
Fig. 1 is a flow chart of an information popularity prediction method based on a double-variation cascade self-encoder according to an embodiment of the present invention.
As shown in fig. 1, the information popularity prediction method based on the double-variation cascade self-encoder comprises the following steps:
s101: acquiring a global interaction graph and a cascade graph in the information diffusion process;
s102: an information diffusion model is built, and a global interaction diagram and a cascade diagram are input into the information diffusion model, wherein the information diffusion model comprises a variation diagram self-encoder and a variation time sequence self-encoder;
s103: and outputting a popularity prediction result of the information through the information diffusion model.
First, the related concepts in the information diffusion theory related to the invention are introduced: social networks, information cascades, and cascade graphs.
Definition 1 social network: refers to a large graph of interactions between users on social media, generally defined as a directed graphWherein->Representing the node set of the diagram, i.e. the set of users in social media, and +.>Representing edge sets of graphs, i.e. interactions or concerns between users, typically directed edges (v, u) ∈ε S Indicating that user v is interested in user u. FIG. 2 is an example of a social network containing unidirectional attention, bidirectional attention, and unassociated new users (orphan nodes u 7).
When a message is posted on the social media platform, others may forward the content, triggering the information dissemination process. Thus, for each message, its information cascade and cascade graph can be defined.
Definition 2 information concatenation: refers to a given piece of information m i Observing a sequence C in which users participating in forwarding (or employing) the information over a period of time order the participants in the sequence i ＝{(v j ，u j ，t j )|t j ≤t p Defined as a cascade of information, wherein each triplet (v j ，u j ，t j ) Indicated at t j Time user u j From user v j Where forwards information m i ，t p Expressed as the observation off-time.
Definition 3 information cascade diagram: given cascade C i Defining its cascade graph as directed graphWherein node set->Comprises cascade C i Is->For the edge set, each directed edge (v j ，u j ) Representing user u j Forward from v j Is a message of (a). An example of some of the information cascade is given in fig. 3, where the orange squares represent the message m i Below which is represented message m i Corresponding cascade diagram->In m 3 And->For example, a->Node setEdge set->
Definition 4 information popularity prediction: given information m i And observation window (0, t) p ]Internally corresponding cascade diagramDefining information popularity prediction problem as prediction slave monitoring time t p The increment of the cascade of information taking place over a period of time T in the future, i.eConsidering that any user in an open social network has the potential to propagate information, all users of the social network are defined as diffusion domains.
Further, in one embodiment of the present invention, inputting the global interaction graph and the cascade graph into the information diffusion model includes:
processing the global interaction graph and the cascade graph respectively to obtain global user embedding and local cascade graph embedding, and connecting the global user embedding and the local cascade graph embedding in series;
using the result of the serial connection of the global user embedding and the local cascade diagram embedding as input, generating a low-dimensional implicit representation of the nodes in the cascade diagram by using a variational diagram self-encoder, and generating a hidden variable representation representing the whole cascade sequence by using a variational time sequence self-encoder;
and (3) inputting the low-dimensional implicit representation and the hidden variable representation into a multi-layer perceptron after being connected in series, and carrying out popularity prediction on the information.
NetSMF is a method for quickly embedding large-scale graphs based on sparse spectrogram technology and matrix decomposition technology. The NetMF matrix is represented by formula (1),
wherein, vol (G) = Σ i ∑ j A ij ，log ° (·) represents taking log element by element.
Because the computation cost of dense matrix decomposition of the large-scale graph is quite high, the graph is thinned and then decomposed on the sparse matrix. The specific flow of NetSMF is shown in algorithm 1.
The idea of random singular value decomposition (algorithm 1 line 10) is: solving the approximation Q epsilon of the original matrix range n×k (k < n), decomposing the randomly initialized matrix Q after multiplication with the original matrix to obtain a stable vector matrix, satisfying A (approximately) QQ T A。B＝Q T A is the transition between matrix a and matrix Q. For matrix B E k×n SVD decomposition is performed, i.eThus A.apprxeq.QQ T A＝Q(S∑V T ) Final A≡U ΣV T And completing approximate singular value decomposition of the 4 matrix. Since this process supports parallel computing, the solution can be done efficiently, and thus has good performance on a large scale map.
The invention utilizes the Graphwave method to learn the structure of the propagation cascade graph and generates the user embedded vector X based on the local topological structure C 。
First, the laplacian matrix (L C ＝D C -A C ) As an algorithm input. Eigenvalues (lambda) decomposed by a Laplace matrix 0 ＜λ 1 ≤...≤λ M ) Corresponding to the signal frequency in signal processing, if the characteristic value is larger, the corresponding characteristic vector will change sensitively along with the change of the topological structure of the graph, and if the characteristic value is smaller, the corresponding characteristic vector change amplitude is smaller, so the heat core g will be in graph wave s (λ)＝e -sλ As a graph filter kernel and designing a thermal kernel from the low-pass modulation effect to generate a smoothed embedded representation vector ψ=ue -sΛ U T 。
Ψ u，s I.e. the embedded vector corresponding to user u, whereinIs psi u，s The mth wavelet coefficient component of (c). The embedded vector elements of each node in the graph are treated as a probability distribution of random variables, which is then represented by an empirical feature function. The characteristic function of the random variable X is defined as +.> Wherein for a given node u and scale parameter s, the empirical characteristic function is defined as +.>Meaning that column average computation is performed (algorithm 2 line 4). Finally splicing the real part and the imaginary part values of the node in the cascade diagram through calculation (algorithm 2 line 6) to obtain the embedding +_of the node u in the cascade diagram>
Further, in one embodiment of the present invention, processing the cascade graph to obtain a local cascade graph embedding includes:
laplacian matrix L of cascade diagram C ＝D C -A C As an algorithm input;
eigenvalue lambda decomposed from Laplace matrix 0 ＜λ 1 ≤...≤λ M Corresponding to the signal frequency in the signal processing: if the feature value is larger, the corresponding feature vector will change sensitively along with the topological structure change of the graph, and if the feature value is smaller, the change amplitude of the corresponding feature vector will be smaller, and the kernel g will be heated s (λ)＝e -sλ As a graph filter kernel and designing a thermal kernel from the low-pass modulation effect to generate a smoothed embedded representation vector ψ=ue -sΛ U T The method comprises the steps of carrying out a first treatment on the surface of the Wherein ψ is u，s I.e. the embedded vector corresponding to user u, whereinIs psi u，s An mth wavelet coefficient component of (2);
the embedded vector elements of each node of the cascade diagram are regarded as probability distribution of a random variable, and the probability distribution is expressed by using an empirical characteristic function, and the characteristic function of the random variable X is defined as Wherein for a given node u and scale parameter s, the empirical feature function is defined as +.>Meaning column mean calculation +.>The real part Re (phi) of the node in the cascade diagram u (t)) and imaginary value Im (phi) u (t)) to obtain the embedding of the node u in the cascade diagram>
To explicitly evaluate whether the final generated node embedded representation better expresses the cascade graph structure, the adjacency matrix A and the feature matrix X of the cascade graph are taken as inputs, and a variable-division graph self-encoder is utilized to generate a low-dimensional implicit representation Z of the nodes in the cascade graph vgae . The module encodes using the graph attention network GAT, and the Decoder reconstructs the original adjacency matrix using an Inner-Product (Inner-Product) approach.
The Encoder portion first generates a high-dimensional gaussian distribution and then samples the gaussian distribution to generate a low-dimensional representation vector of the corresponding node. The GAT-based encoding operation is shown in equation (2),
Z vgae ＝GAT_Encoder(X，A) (2)
in the formula (2), the amino acid sequence of the formula (2),representing node embedding matrix in cascade diagram, < >>Adjacency matrix representing cascade diagram, < >>The node low-dimensional implicit embedding matrix representing the gat_encoder output. Taking node i as an example, the corresponding initial feature vector +.>And its neighborhood as input, GAT of the first layer 1 Generating a first intermediate hidden layer representation h i E d, next parallel GAT 2 And GAT 3 Respectively generating the mean value +.about.of Gaussian distribution corresponding to the node i>Variance from log treated +.>Wherein each dimension corresponds to the gaussian distribution of each dimension in the low-dimensional embedded representation of node i. Where log sigma is selectively generated 2 And not directly generating σ, because the graph attention network output value cannot be controlled within a positive number range. The operation of GAT from a node point of view is shown in equation (4),
in the formula (4), i represents a tandem operation. In order to stabilize the learning process, the drawing attention network adopts a multi-head attention mechanism, training results of multi-head attention are connected in series, and K represents the number of multi-head attention. Assuming node i is the central node of the cascade graph (i.e. the user node that issued the message),then represent its neighbor node, W (k) x j Representing an embedded representation of node j after linear transformation, whereas +.>Representing the attention coefficient of the node j to the node i in the kth attention head, the calculation formula of which is shown as (5),
the Gaussian distribution obeyed by the sample is obtained after the operationNext a new node vector needs to be sampled +.>Taking into account the calculation of the sampling gradientThe invention adopts a Reparameterization (Reparameterization) method, which firstly samples a +.>Then generating z according to equation (6) i 。
z i ＝μ i +∈ i σ i (6)
The Decoder portion reconstructs the adjacency matrix by inner product of inter-node hidden variable representationsAs shown in formula (7), wherein σ ()'s represent nonlinearity
The function is activated, here using Sigmoid.
Further, the loss calculation is performed by a cross entropy loss function, as shown in equation (8),
where y.epsilon. 0|1 represents the value of a certain element in the adjacency matrix A,representing the values of the elements in the reconstructed adjacency matrix corresponding to the a-position. Meanwhile, in order to ensure the variation characteristic, the KL divergence is often introduced in the variation self-encoder loss calculation, and the KL divergence can measure the similarity between the conditional distribution and the prior distribution of the hidden variable. The loss calculation of the final VGAE part is shown in equation (9),
wherein KL [ q (Z) vgae |X，A)||p(H)]Is KL divergence, where q (Z vgae X, a) represents the distribution calculated by GAT and p (H) represents the a priori distribution, where a standard normal distribution is chosen. Proved that the KL divergence can be calculated by the expression (10),
generating structural hidden variable representation Z of each node in cascade graph based on VGAE vgae It is further desirable to generate a structure hidden variable representation through a pooling operation that can represent the entire cascading graph. Considering that different users play different roles in cascade propagation, the invention selects SAGGool method to complete weighted summation pooling, and generates hidden variable representation of the whole cascade diagram
The SAGGool method improves the TopK pooling mode based on spectral domain convolution, and pools the information of the topological structure and the node characteristics of the comprehensive graph. The specific operation comprises three steps: first, constructing a self-attention strickling layer, obtaining self-attention scores of each node in the cascade graph on the basis of graph convolution, wherein the calculation process is shown as a formula (11), and the calculation process is shown as a formula (11), wherein Θ att ∈ N×1 Represented as a projective transformation matrix, Z (l+1) ∈ N×1
Expressed as a learned self-attention score, with a value between [ -1,1 ]; σ () represents a nonlinear activation function, here using the tanh function.
And secondly, sorting according to the self-attention scores of the nodes in the graph, selecting TopK nodes at a pooling rate K to form a Mask matrix, and updating the subgraph by using the Mask matrix. The calculation process is shown in the formula (12),
and thirdly, splicing the Readout layer by adopting two modes of average pooling and maximum pooling to generate a final representation vector. The calculation formula is shown as formula (13), wherein,representing the pooled output vector.
After pooling, a structure hidden variable representation representing the entire cascade graph is generated
Further, in one embodiment of the present invention, the generating a low-dimensional implicit representation of nodes in a cascade graph from an encoder using a variogram takes as input the result of concatenating a global user-insert and a local cascade graph insert, comprising:
the graph attention network GAT is adopted for coding, and the original adjacency matrix is reconstructed in an inner product mode.
The present invention focuses on modeling the cascading timing relationship, and therefore employs an improved variational timing self-encoder to learn the cascading sequence and generate a hidden variable representation that represents the entire cascading sequence.
Similar to VGAE, the Encoder portion of VTAE first generates each intermediate implicit embedding with a bi-directional GRU network, then generates a Gaussian distribution with a DNN network on this basis, and finally samples on the Gaussian distribution to generate an hidden variable representation of the corresponding node. The whole operation is shown as the formula (14) and the formula (1 5),
Z vtae ＝RNN_Encoder(X) (14)
wherein the method comprises the steps of，Embedded representation representing bi-directional learning to user i, mean +.>And variance ofIs based on two relatively independent DNNs. Similarly, a low-dimensional embedded representation corresponding to each user in the cascade sequence is generated based on a Reparameterization (Reparameterization) method>
The Decoder section uses a layer of bi-directional GRU network to restore the representation vector of each user, as shown in equations (16) and (17),
wherein,representing the restored output vector.
VTAE measures model loss by measuring the difference between input and output, and as such, VTAE requires adding KL divergence in the loss calculation to prevent model overfitting, whose calculation formulas are shown in (18) - (20),
generating a hidden variable representation of each user in the cascade sequence based on the VTAE further requires deriving a hidden variable representation representing the entire cascade sequence by a Readout operation. Considering the time attenuation effect in information propagation, the invention reads the final output by using the GRU module and uses the time attenuation coefficient as weight to carry out weighted summation pooling, wherein the process is shown as a formula (21), and z j Representing the correspondence of the user
The hidden variable represents the value of the hidden variable,indicating a time stamp of t j The attenuation coefficient at the moment is calculated as f (T-T j )＝l，T-t j ∈[t l-1 ，t l )。R T Representing the cascade length over the observation time.
After pooling, a time-sequential hidden variable representation representing the entire concatenated sequence is generated
Further, in one embodiment of the invention, generating a hidden variable representation representing the entire concatenated sequence from the encoder using the variant timing comprises:
generating each intermediate implicit embedding with a bi-directional GRU network;
generating Gaussian distribution by using a DNN network;
sampling on the gaussian distribution generates a hidden variable representation of the corresponding node.
Will beAnd->After being connected in series, the signals are input into a multi-layer perceptron, and the network structure is shown in figure 4.
The final output predicted value isCalculating the difference between the predicted value and the true value by using MSELoss, and setting the parameter lambda 1 As a structural loss corresponding to VGAE, a parameter lambda is set 2 As a timing loss corresponding to VTAE.
Further, in one embodiment of the present invention, constructing an information diffusion model includes:
the information diffusion model is optimized using the loss function, expressed as:
wherein lambda is 1 Lambda is the structural loss 2 Is a timing penalty.
The embodiment of the invention provides an information popularity prediction method based on a double-variation cascade self-encoder, which provides a double-variation cascade self-encoder (DVCAE), enhances the generalization and the robustness of a prediction model by utilizing the generating capacity of the variation self-encoder, models the cascaded graph structure and time sequence change in parallel by utilizing the variation self-encoder, and optimizes the prediction loss, the structure reconstruction loss and the time sequence reconstruction loss in a combined way. The model not only has good prediction performance, but also can explicitly evaluate the learning of the model on the cascade graph structure and the learning of the time sequence characteristics, and ensures that the input characteristics are effectively compressed. According to visualization, ablation and parameter sensitivity experimental analysis, the correlation between the structural hidden variable representation and the information popularity is high, and the inherent mode of information diffusion can be well learned through modeling the propagation diagram structure, so that the future information popularity can be accurately predicted.
In order to realize the embodiment, the invention also provides an information popularity prediction device based on the double-variation cascade self-encoder.
Fig. 5 is a schematic structural diagram of an information popularity prediction device based on a double-variation cascade self-encoder according to an embodiment of the present invention.
As shown in fig. 5, the information popularity prediction apparatus based on the double-variation cascade self-encoder includes: an acquisition module 100, an input module 200, an output module 300, wherein,
the acquisition module is used for acquiring a global interaction graph and a cascade graph in the information diffusion process;
the input module is used for constructing an information diffusion model and inputting the global interaction diagram and the cascade diagram into the information diffusion model, wherein the information diffusion model comprises a variation diagram self-encoder and a variation time sequence self-encoder;
and the output module is used for outputting the popularity prediction result of the information through the information diffusion model.
To achieve the above object, an embodiment of a third aspect of the present invention provides a computer device, which is characterized by comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the information popularity prediction method based on the double variation cascade self-encoder as described above when executing the computer program.
To achieve the above object, a fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the information popularity prediction method based on a double variation cascade self-encoder as described above.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (9)
1. The information popularity prediction method based on the double-variation cascade self-encoder is characterized by comprising the following steps of:
acquiring a global interaction graph and a cascade graph in the information diffusion process;
an information diffusion model is built, the global interaction graph and the cascade graph are input into the information diffusion model, and the information diffusion model comprises a variation graph self-encoder and a variation time sequence self-encoder;
and outputting a popularity prediction result of the information through the information diffusion model.
2. The method of claim 1, wherein said inputting the global interaction map and cascade map into an information diffusion model comprises:
processing the global interaction graph and the cascade graph respectively to obtain a global user embedding and a local cascade graph embedding, and connecting the global user embedding and the local cascade graph embedding in series;
using the result of the serial connection of the global user embedding and the local cascade diagram embedding as input, generating a low-dimensional implicit representation of nodes in the cascade diagram by using the variogram self-encoder, and generating a hidden variable representation representing the whole cascade sequence by using the variogram self-encoder;
the low-dimensional implicit representation is connected with the hidden variable representation in series and then is input to a multi-layer perceptron to conduct popularity prediction of the information.
3. The method of claim 2, wherein processing the cascade graph to obtain a local cascade graph embedding comprises:
laplacian matrix L of the cascade graph C ＝D C -A C As an algorithm input;
eigenvalue lambda decomposed from the laplace matrix 0 <λ 1 ≤...≤λ M Corresponding to the signal frequency in the signal processing: if the characteristic value is larger, the corresponding characteristic vector is sensitive to change along with the topological structure change of the graph, and if the characteristic value is smaller, the corresponding characteristic vector is smaller in change amplitude, and the kernel g is heated s (λ)＝e -sλ As a graph filter kernel and designing a thermal kernel from the low-pass modulation effect to generate a smoothed embedded representation vector ψ=ue -sΛ U T The method comprises the steps of carrying out a first treatment on the surface of the Wherein ψ is u,s I.e. the embedded vector corresponding to user u, whereinIs psi u,s An mth wavelet coefficient component of (2);
the embedded vector elements of each node of the cascade diagram are regarded as a probability distribution of random variables, and the distribution is then represented by an empirical characteristic function, and the characteristic function of the random variable X is defined asWherein,for a given node u and scale parameter s, the empirical feature function is defined as +.>Meaning column mean calculation +.>The real part Re (phi) of the node in the cascade diagram u (t)) and imaginary value Im (phi) u (t)) to obtain the embedding of the node u in the cascade diagram>
4. The method of claim 2, wherein the concatenating the global user embedding and the local cascade map embedding results as input, generating a low-dimensional implicit representation of nodes in the cascade map from an encoder using the variogram, comprises:
the graph attention network GAT is adopted for coding, and the original adjacency matrix is reconstructed in an inner product mode.
5. The method of claim 2, wherein the generating, with the variability sequential self-encoder, a hidden variable representation representing the entire concatenated sequence comprises:
generating each intermediate implicit embedding with a bi-directional GRU network;
generating Gaussian distribution by using a DNN network;
sampling on the gaussian distribution generates a hidden variable representation of the corresponding node.
6. The method of claim 1, wherein the constructing an information diffusion model comprises:
optimizing the information diffusion model using the loss function is expressed as:
wherein lambda is 1 Lambda is the structural loss 2 Is a timing penalty.
7. An information popularity prediction device based on a double-variation cascade self-encoder is characterized by comprising the following modules:
the acquisition module is used for acquiring a global interaction graph and a cascade graph in the information diffusion process;
the input module is used for constructing an information diffusion model, and inputting the global interaction graph and the cascade graph into the information diffusion model, wherein the information diffusion model comprises a variational graph self-encoder and a variational time sequence self-encoder;
and the output module is used for outputting the popularity prediction result of the information through the information diffusion model.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of information popularity prediction based on a double-variation cascade self-encoder as claimed in any one of claims 1-6 when executing the computer program.
9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the information popularity prediction method based on a bi-variational cascade self-encoder as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311502493.4A CN117610717A (en) | 2023-11-13 | 2023-11-13 | Information popularity prediction method based on double-variation cascade self-encoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311502493.4A CN117610717A (en) | 2023-11-13 | 2023-11-13 | Information popularity prediction method based on double-variation cascade self-encoder |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117610717A true CN117610717A (en) | 2024-02-27 |
Family
ID=89948775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311502493.4A Pending CN117610717A (en) | 2023-11-13 | 2023-11-13 | Information popularity prediction method based on double-variation cascade self-encoder |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117610717A (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544666A (en) * | 2018-10-26 | 2019-03-29 | 中国科学院计算技术研究所 | A kind of full automatic model deformation transmission method and system |
| US20210019598A1 (en) * | 2019-07-18 | 2021-01-21 | International Business Machines Corporation | Recurrent autoencoder for chromatin 3d structure prediction |
| CN112733027A (en) * | 2021-01-08 | 2021-04-30 | 西安交通大学 | Hybrid recommendation algorithm based on local and global representation model joint learning |
| CN112906770A (en) * | 2021-02-04 | 2021-06-04 | 浙江师范大学 | Cross-modal fusion-based deep clustering method and system |
| CN112987675A (en) * | 2021-05-06 | 2021-06-18 | 北京瑞莱智慧科技有限公司 | Method, device, computer equipment and medium for anomaly detection |
| CN113139134A (en) * | 2021-04-26 | 2021-07-20 | 中国人民解放军国防科技大学 | Method and device for predicting popularity of user-generated content in social network |
| CN113870040A (en) * | 2021-09-07 | 2021-12-31 | 天津大学 | Double-flow graph convolution network microblog topic detection method fusing different propagation modes |
| CN114065996A (en) * | 2021-04-02 | 2022-02-18 | 四川省计算机研究院 | Traffic flow prediction method based on variational self-coding learning |
| CN115618124A (en) * | 2022-09-28 | 2023-01-17 | 华南理工大学 | Propagation popularity prediction method for bidirectional social influence learning |
| CN115759482A (en) * | 2022-12-29 | 2023-03-07 | 人民网股份有限公司 | Social media content propagation prediction method and device |
| US20230083437A1 (en) * | 2021-08-27 | 2023-03-16 | The Regents Of The University Of California | Hyperdimensional learning using variational autoencoder |
| CN116189062A (en) * | 2023-04-21 | 2023-05-30 | 北京金楼世纪科技有限公司 | Video popularity prediction method and system |
| CN116307022A (en) * | 2022-10-12 | 2023-06-23 | 山东省计算中心（国家超级计算济南中心） | Public opinion hotspot information prediction method and system |
| CN116308854A (en) * | 2022-08-30 | 2023-06-23 | 电子科技大学 | Information cascading popularity prediction method and system based on probability diffusion |
| CN116975778A (en) * | 2023-07-28 | 2023-10-31 | 郑州大学 | Social network information propagation influence prediction method based on information cascading |
-
2023
- 2023-11-13 CN CN202311502493.4A patent/CN117610717A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544666A (en) * | 2018-10-26 | 2019-03-29 | 中国科学院计算技术研究所 | A kind of full automatic model deformation transmission method and system |
| US20210019598A1 (en) * | 2019-07-18 | 2021-01-21 | International Business Machines Corporation | Recurrent autoencoder for chromatin 3d structure prediction |
| CN112733027A (en) * | 2021-01-08 | 2021-04-30 | 西安交通大学 | Hybrid recommendation algorithm based on local and global representation model joint learning |
| CN112906770A (en) * | 2021-02-04 | 2021-06-04 | 浙江师范大学 | Cross-modal fusion-based deep clustering method and system |
| CN114065996A (en) * | 2021-04-02 | 2022-02-18 | 四川省计算机研究院 | Traffic flow prediction method based on variational self-coding learning |
| CN113139134A (en) * | 2021-04-26 | 2021-07-20 | 中国人民解放军国防科技大学 | Method and device for predicting popularity of user-generated content in social network |
| CN112987675A (en) * | 2021-05-06 | 2021-06-18 | 北京瑞莱智慧科技有限公司 | Method, device, computer equipment and medium for anomaly detection |
| US20230083437A1 (en) * | 2021-08-27 | 2023-03-16 | The Regents Of The University Of California | Hyperdimensional learning using variational autoencoder |
| CN113870040A (en) * | 2021-09-07 | 2021-12-31 | 天津大学 | Double-flow graph convolution network microblog topic detection method fusing different propagation modes |
| CN116308854A (en) * | 2022-08-30 | 2023-06-23 | 电子科技大学 | Information cascading popularity prediction method and system based on probability diffusion |
| CN115618124A (en) * | 2022-09-28 | 2023-01-17 | 华南理工大学 | Propagation popularity prediction method for bidirectional social influence learning |
| CN116307022A (en) * | 2022-10-12 | 2023-06-23 | 山东省计算中心（国家超级计算济南中心） | Public opinion hotspot information prediction method and system |
| CN115759482A (en) * | 2022-12-29 | 2023-03-07 | 人民网股份有限公司 | Social media content propagation prediction method and device |
| CN116189062A (en) * | 2023-04-21 | 2023-05-30 | 北京金楼世纪科技有限公司 | Video popularity prediction method and system |
| CN116975778A (en) * | 2023-07-28 | 2023-10-31 | 郑州大学 | Social network information propagation influence prediction method based on information cascading |
Non-Patent Citations (4)
| Title |
|---|
| 敬鑫: "基于图网络和持续学习的信息级联规模研究", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 1, 15 January 2023 (2023-01-15) * |
| 李树青等: "基于显隐式信息融合和单类协同过滤方法的主题词推荐", 《图书情报工作》, vol. 67, no. 3, 20 February 2023 (2023-02-20) * |
| 李飞: "基于图神经网络的混合推荐系统研究", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 1, 15 January 2023 (2023-01-15) * |
| 王雪婷: "社交网络信息级联预测模型及应用", 《中国优秀硕士学位论文全文数据库基础科学辑》, no. 1, 15 January 2023 (2023-01-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | # exploration: A study of count-based exploration for deep reinforcement learning | |
| Chen et al. | E-LSTM-D: A deep learning framework for dynamic network link prediction | |
| Bui et al. | Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues | |
| Liu et al. | Graph normalizing flows | |
| Zhang et al. | Privacy-preserving double-projection deep computation model with crowdsourcing on cloud for big data feature learning | |
| CN111541570B (en) | Cloud service QoS prediction method based on multi-source feature learning | |
| CN111651671B (en) | User object recommendation method, device, computer equipment and storage medium | |
| CN113536144B (en) | Social network information propagation scale prediction method and device | |
| CN113868474A (en) | Information cascade prediction method based on self-attention mechanism and dynamic graph | |
| Ardalani-Farsa et al. | Residual analysis and combination of embedding theorem and artificial intelligence in chaotic time series forecasting | |
| Sheng et al. | Enhanced graph neural network for session-based recommendation | |
| CN110232151B (en) | Construction method of QoS (quality of service) prediction model for mixed probability distribution detection | |
| Yang et al. | Privacy‐preserving generative framework for images against membership inference attacks | |
| Louis et al. | Simplifying subgraph representation learning for scalable link prediction | |
| Qamar et al. | Artificial neural networks: An overview | |
| Gurung et al. | Decentralized quantum federated learning for metaverse: Analysis, design and implementation | |
| CN115834251B (en) | Hypergraph-transform-based threat hunting model building method | |
| Xu et al. | Collective vertex classification using recursive neural network | |
| Luan et al. | LRP‐based network pruning and policy distillation of robust and non‐robust DRL agents for embedded systems | |
| CN117610717A (en) | Information popularity prediction method based on double-variation cascade self-encoder | |
| Dash | DECPNN: A hybrid stock predictor model using Differential Evolution and Chebyshev Polynomial neural network | |
| CN113537613B (en) | Temporal network prediction method for die body perception | |
| LIN et al. | Deephgnn: A novel deep hypergraph neural network | |
| CN116307022A (en) | Public opinion hotspot information prediction method and system | |
| Ward et al. | A practical guide to graph neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |