CN111949892A - Multi-relation perception temporal interaction network prediction method - Google Patents
Multi-relation perception temporal interaction network prediction method Download PDFInfo
- Publication number
- CN111949892A CN111949892A CN202010797094.5A CN202010797094A CN111949892A CN 111949892 A CN111949892 A CN 111949892A CN 202010797094 A CN202010797094 A CN 202010797094A CN 111949892 A CN111949892 A CN 111949892A
- Authority
- CN
- China
- Prior art keywords
- interaction
- relationship
- representation
- neighbor
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000009625 temporal interaction Effects 0.000 title claims abstract description 37
- 230000008447 perception Effects 0.000 title claims abstract description 15
- 230000003993 interaction Effects 0.000 claims abstract description 194
- 230000002452 interceptive effect Effects 0.000 claims abstract description 61
- 238000004220 aggregation Methods 0.000 claims abstract description 22
- 230000002776 aggregation Effects 0.000 claims abstract description 17
- 238000005065 mining Methods 0.000 claims abstract description 5
- 239000011159 matrix material Substances 0.000 claims description 27
- 238000004364 calculation method Methods 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 22
- 230000007246 mechanism Effects 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 10
- 230000000644 propagated effect Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 230000014509 gene expression Effects 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 230000000306 recurrent effect Effects 0.000 claims description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 4
- 238000009795 derivation Methods 0.000 claims description 4
- 238000012935 Averaging Methods 0.000 claims description 2
- 125000004122 cyclic group Chemical group 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 230000003068 static effect Effects 0.000 description 3
- 230000002123 temporal effect Effects 0.000 description 3
- 238000007792 addition Methods 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9536—Search customisation based on social or collaborative filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a multi-relation perception temporal interaction network prediction method, which comprises the following steps: (1) taking the interaction in the temporal interaction network as a sample; (2) processing each interaction in sequence according to the interaction occurrence time, mining nodes with historical interaction relation, common interaction relation and interaction sequence similarity relation between the nodes based on historical interaction information, and constructing a local relation graph before current interaction for the interaction nodes; (3) predicting the representation of the object before the current interaction according to the representation of the user after the last interaction and the representation of the user based on the neighbor obtained by hierarchical multi-relation perception aggregation; (4) updating the representation of the interactive node according to the representation of the interactive node after the last interaction, the time interval of the last interaction and the current interaction and the representation based on the neighbor; (5) after the temporal interaction network prediction model is trained, the temporal interaction network prediction model with optimized parameters is used for predicting articles which are likely to interact with the user.
Description
Technical Field
The invention relates to the field of temporal interaction network prediction, in particular to a multi-relation perception temporal interaction network prediction method.
Background
In many areas of real life, such as e-commerce (customers purchasing goods), education platforms (students attending a course of a mu class), and social network platforms (users posting posts in a community), users interact with different items at different times, and the interaction between the users and the items forms a temporal interaction network. Temporal interactive networks increase the concern of interaction time compared to static interactive networks. The temporal interaction network prediction refers to predicting which article a user interacts with before interaction occurs, and has significance for tasks such as commodity recommendation, course recommendation and community recommendation.
The existing prediction method based on the temporal interaction network comprises two types, one type is a prediction method not based on a graph structure, and the other type is a prediction method based on a graph structure. The prediction method not based on the graph structure is a prediction method based on a hidden semantic model and a prediction method based on a sequence model, which means that the interaction between the user and the article is not represented in the graph structure but in other forms such as a matrix or a sequence. The prediction method based on the latent semantic model introduces time information on the basis of the traditional latent semantic model to model the change of user interest and article attributes, and obtains the representation of the user and the articles for prediction. However, this type of work does not take into account the order in which interactions between the user and the item occur. In a temporal interactive network, abundant sequence information often exists, and in order to utilize the information, a plurality of prediction methods based on a sequence model are proposed, however, the methods use a static representation of an article as an input to update the representation of a user, and ignore the current state information of the article. In addition, most of these methods only consider dynamic changes in user interests, ignoring dynamic changes in item attributes.
In order to mine more abundant information in user and article interaction, many prediction methods based on graph structure are proposed. Although the traditional prediction method based on the graph structure takes the time period as the node in the graph, the traditional prediction method is still a static graph in nature and cannot well model the dynamics of the user and the property of the article. To solve this problem, many prediction methods based on temporal interaction network embedding are proposed. The prediction method based on the temporal interaction network embedding is used for embedding the temporal interaction network to obtain the representation of the user and the article so as to predict. The prediction method based on the temporal interactive network embedding may be classified into a prediction method considering no neighbor information and a prediction method considering neighbor information according to whether neighbor information is aggregated at the time of embedding. The prediction method without considering the neighbor information ignores the influence of the neighbor information although modeling the attribute change of the interactive node. When the neighbor information is considered in the conventional prediction method considering the neighbor information, only the nodes with the historical interaction relationship are used as the neighbor nodes, and other relationship types (common interaction relationship, interaction sequence similarity relationship and the like) in the historical interaction information are ignored.
Disclosure of Invention
In view of the above, the invention provides a multi-relationship-aware temporal interaction network prediction method, which improves the accuracy of temporal interaction network prediction by effectively utilizing neighbor information.
The technical scheme of the invention is as follows:
a multi-relation-aware temporal interaction network prediction method comprises the following steps:
(1) with user uiAnd an article vjInteraction (u) occurring at time ti,vjT) constructing a training data set as a sample, and batching the training data set;
(2) for interaction (u)i,vjT), mining the nodes with historical interaction relationship, common interaction relationship and interaction sequence similarity relationship between the nodes based on historical interaction information to obtain interactive nodes uiAnd vjConstructing a local relationship graph before current interaction
And
(3) according to a local relationship diagram
And
obtaining user u through hierarchical multi-relation perception aggregationiNeighbor-based representation
And an article vjNeighbor-based tableDisplay device
(4) According to user uiLast interactive representation
And user uiNeighbor-based representation
Calculating current pre-interaction item v by utilizing full connection layerjRepresentation of predictions
(5) According to user uiAnd an article vjLast interactive representation
And
time interval between last interaction and current interaction
And
and neighbor-based representation
And
respectively calculating user u by utilizing two recurrent neural network layersiAnd an article vjCurrently interacted with representation
And
(6) according to the current pre-interaction item vjRepresentation of predictions
And a real representation
Error between, user uiRegularization loss and article vjRegularization loss, calculating the overall loss
According to the loss of all samples in the batch
Adjusting network parameters in a temporal interaction network prediction model until all batches participate in model training, wherein the temporal interaction network prediction model comprises all full connection layers and a cyclic neural network layer used in the steps (2) to (6);
(7) and predicting the articles which are possibly interacted by the user by using the temporal interaction network prediction model after the parameters are adjusted.
According to the method, the multi-relation among the nodes is mined based on historical interaction information, a local relation graph before current interaction is constructed for the interaction nodes, and interaction influence of neighbor nodes propagated according to different relation types is considered through hierarchical multi-relation perception aggregation. Compared with the prior art, the method has the advantages that:
1) the method comprises the steps of mining nodes with historical interaction relation, common interaction relation and interaction sequence similarity relation between the nodes and interaction nodes based on historical interaction information, constructing a local relation graph before current interaction for the interaction nodes, obtaining the representation of the interaction nodes based on neighbors through hierarchical multi-relation perception aggregation to predict the representation of articles and update the representation of the interaction nodes, considering the multi-relation between the nodes, and effectively utilizing neighbor information, so that the accuracy of the prediction of a temporal interaction network is improved;
2) and introducing a graph neural network with an attention layer, endowing corresponding weights to the neighbor nodes according to the interaction influence propagated from the neighbor nodes and the relationship type between the nodes, and hierarchically aggregating the interaction influence propagated according to different relationship types.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flowchart illustrating an overall method for predicting a temporal interaction network based on multi-relationship awareness according to an embodiment;
FIG. 2 is a block diagram of an overall framework of a temporal interaction network prediction method for multi-relationship awareness according to an embodiment;
fig. 3 is a schematic diagram of hierarchical multi-relationship-aware aggregation provided by an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is an overall flowchart of a temporal interaction network prediction method based on multi-relationship awareness according to an embodiment. Fig. 2 is an overall framework diagram of a temporal interaction network prediction method based on multi-relationship awareness according to an embodiment. As shown in fig. 1 and fig. 2, a multi-relationship-aware temporal interaction network prediction method provided by an embodiment includes the following steps:
step 1, inputting a temporal interaction network
Representing N interactions in time order, i being an index of the interactions, taking each interaction s as a sample to obtain a training data set, where s ═ v, t represents the user
And an article
In that
The interaction that occurs at the moment of time,
and
respectively a user set, an item set and an interaction time set. And (4) batching the training data set according to a t-n-Batch algorithm, wherein the total number of batches is C.
In an embodiment, the training data set is batched by using a t-n-Batch algorithm, so that the interactions in the same Batch can be processed in parallel, and the time dependence between the interactions can be kept when all batches are processed according to the index sequence of the batches.
The process of batching the training data set using the t-n-Batch algorithm is:
first, N empty batches are initialized, and then the training data set is traversed, dividing each interaction into a respective batch. Let lastU and lastV record the maximum index of the batch where the user and item are located, respectively. To interact with (u)i,vjT) is for example lastU ui]Representing user uiThe maximum index of the lot, i.e., the index is lastU ui]The interaction in the batch of (1) involves user uiAnd the batch is the largest index of the batches related to the user. In the same way, lastVj]Representing an article vjMaximum index of the lot in which idxN is user uiAnd an article vjThe maximum index of the batch where all the neighbor nodes are located. Because each node can only appear once in a batch at most, and the ith interaction and the (i + 1) th interaction of each node need to be divided into the kth batch B respectivelykAnd the first batch BlWherein k is<l, thus interacting(ui,vjT) will be divided into indices max (lastU u)i],lastV[vj]Batch of idxN) + 1. And after the batch division is finished, removing redundant empty batches, wherein the total number of the residual batches is C.
And 2, sequentially selecting a batch of training samples with the index of k from the training data set, wherein k belongs to {1,2, …, C }. For each training sample in the batch, steps 3-7 are performed.
Step 3, for interaction (u)i,vjT), mining the nodes with historical interaction relationship, common interaction relationship and interaction sequence similarity relationship between the nodes based on historical interaction information to obtain interactive nodes uiAnd vjConstructing a local relationship graph before current interaction
And
in this embodiment, the node n is usediFor example, a local relationship diagram
Wherein
And
respectively represent and node niA set of related nodes, a set of edges, a set of relationship types, and a set of relationship attributes. Edge e is defined as a triple
Representing a node niAnd node njThere is a relationship between them, the relationship type is
Comprises three types of historical interactive relationship, common interactive relationship and interactive sequence similarity relationship, and the relationship attribute is
Where q is (t, w), t denotes a time attribute, and w denotes a weight attribute.
The specific method of multi-relation derivation is as follows:
1) historical interaction relationships
If two nodes have interacted historically, a historical interaction relationship exists between the two nodes, the time attribute t of the historical interaction relationship is the last interaction time of the two nodes, and the weight attribute w is the historical interaction frequency.
2) Mutual interaction relation
If two nodes interact with the same node in the T time period, a common interaction relationship exists between the two nodes. The time attribute t of the common interaction relationship is the time of the last common interaction of the two nodes, wherein the time of the common interaction is the closest time to the current time in the time of the interaction of the two nodes and the same node, and the weight attribute w is the historical common interaction times.
3) Interaction sequence similarity relationship
All the interactive sequences are regarded as 'documents', each interactive sequence is regarded as 'sentences', nodes in the interactive sequences are regarded as 'words', and after the user interactive sequences and the article interactive sequences are respectively embedded by using a Doc2Vec model, the representation of each user based on the interactive sequences and the representation of each article based on the interactive sequences can be obtained.
As the interaction between the user and the article continuously occurs, the Doc2Vec model is updated in an incremental training mode, and a new representation of the user and the article based on the interaction sequence is obtained. Given two nodes of the same type (two users or two items) niAnd njPresentation based on interaction sequences
And
calculating the cosine similarity between the two, wherein the calculation mode is as follows:
where, denotes a dot product.
Setting a threshold value mu only when the cosine similarity cosSim
And when the value is larger than the threshold value mu, the interaction sequence similarity relation exists between the two nodes. The time attribute t of the interaction sequence similarity relation is the moment of interaction occurring at the last time in the two node interaction sequences, and the weight attribute w is cosine similarity.
After the nodes with historical interaction relationship, common interaction relationship and interaction sequence similarity relationship with the interaction nodes are mined by the multi-relationship derivation concrete method, the interaction nodes u can be regarded as interaction nodes uiAnd vjConstructing a local relationship graph before current interaction
And
step 4, according to the local relation graph
And
obtaining user u through hierarchical multi-relation perception aggregationiNeighbor-based representation
And an article vjNeighbor-based representation
In an embodiment, the hierarchical multi-relationship aware aggregation comprises two layers of aggregation processes: intra-relationship aggregation and inter-relationship aggregation. Fig. 3 shows a hierarchical multi-relationship perception aggregation diagram.
In order to simplify the operation, the representation of the last interaction of the neighbor nodes is used as the interaction influence propagated by the neighbor nodes. With node niFor example, a local relationship graph of the node before the current interaction is constructed
If node niAs a user, the corresponding user is ujThen the last interactive representation of the node is
If node niIs an article, corresponding to the article vjThen the last interactive representation of the node is
To simplify notation, node n is assignediThe representation after the last interaction is recorded as
When node niWhen interaction occurs, the node is given a local relationship graph in the time interval between the last interaction and the current interaction
The interaction influence propagated by the neighbor node in which the interaction occurs, namely the representation of the neighbor node after the interaction occurs
Wherein M is the number of nodes interacted among the neighbor nodes, and the specific process of hierarchical multi-relationship perception aggregation is as follows:
the first layer is intra-relationship aggregation, and aggregates the interaction influence propagated by neighbor nodes according to the same relationship type, and gives corresponding weights to different neighbor nodes to obtain neighbor representation of the node based on a specific relationship type. To distinguish the type of relationship between nodes, three multi-headed attention comprising K heads with different parameters are usedThe mechanism respectively carries out intra-relationship aggregation on the historical interaction relationship, the common interaction relationship and the interaction sequence similarity relationship to obtain a node niNeighbor representation based on historical interaction relationships
Neighbor representation based on common interaction relationships
And neighbor representation based on inter-sequence similarity
For a given node niIs a neighbor node njThe input of the multi-head attention mechanism is
Then the input of attention mechanism of the kth head
The calculation is as follows:
wherein,
representing a matrix of k-th head input parameters, different relation types
The same is true. According to the input of the neighbor node
The attention coefficient for the kth head is calculated as follows:
wherein,
attention weight matrix representing kth head, different relation types
Different.TDenotes matrix transpose, | denotes vector join operation.
The weight associated with the relationship attribute q is represented and the calculation process is shown in equation (4).
Inputting the relation attribute q ═ t, w into the full-link layer to obtain an output value
If the relationship type r is a history interactive relationship, the t attribute represents the node niAnd a neighbor node njAt the last interaction time, the w attribute represents the historical interaction times of the two nodes; if the relationship type r is a common interaction relationship, the t attribute is the time of the last common interaction of the two nodes, and the w attribute is the historical common interaction times; if the relationship type r is an interaction sequence similarity relationship, the t attribute is the moment of interaction occurring at the last in the interaction sequences of the two nodes, and the w attribute is cosine similarity. The calculation formula is as follows:
wherein WfeatParameter matrix being a fully connected layer, bfeatFor biasing of the fully-connected layer, different heads of different relationship types share the fully-connected layer.
For a given node niAll the neighbor nodes with the relation types of the historical interaction relation are subjected to weight normalization to obtain neighbor nodes njNormalized kth head attention coefficient:
wherein,
is given node niThe relationship type is a neighbor node set of historical interaction relationship. Based on the above calculation, the hidden vector of the k-th head
The calculation is as follows:
for a given node niImplicit vectors obtained by K heads
Obtaining a node n after averagingiNeighbor representation based on historical interaction relationships
The calculation process is shown in formula (7):
the related parameters with the relationship type of the historical interactive relationship are converted into the related parameters of the common interactive relationship, and the neighbor expression based on the common interactive relationship is obtained by using the formulas (2) to (7)
Similarly, the related parameters with the relationship type of the historical interaction relationship are converted into the related parameters of the interaction sequence similarity relationship, and the neighbor expression based on the interaction sequence similarity relationship is obtained by using the formulas (2) to (7)
ToThe related parameters comprise a relationship attribute q, a neighbor node set of each relationship type and an attention weight matrix
The second layer is inter-relationship aggregation, and because the importance of the interaction influence propagated according to different relationship types on a given node is different, corresponding weights are given to different relationship types by using a self-attention mechanism. Given node niNeighbor expressions based on different relation types can be obtained by utilizing intra-relation aggregation, and the neighbor expressions are aggregated through an attention-free mechanism to obtain the node niA neighbor-based representation.
For node niNeighbor representation based on historical interaction relationships
Neighbor representation based on common interaction relationships
And neighbor representation based on inter-sequence similarity
Obtaining input of self-attention mechanism after splicing
The calculation process of the query matrix Q, the key matrix K and the value matrix V of the self-attention mechanism is as follows:
Q＝HWQ (8)
K＝HWK (9)
V＝HWV (10)
wherein
And
respectively, query weight matrix, key weight matrix, and value weightsAnd (4) a heavy matrix. Output of self-attention mechanism
As shown in formula (11):
wherein,
is a scale factor, dk＝dv。
Inputting the output Z of the attention mechanism into the full-connection layer to obtain a node niNeighbor-based representation
The calculation process is shown in formula (12):
wherein, WoutParameter matrix being a fully connected layer, boutIs the bias of the fully connected layer.
According to a local relationship diagram
And
obtaining user u by utilizing the hierarchical multi-relation perception aggregationiNeighbor-based representation
And an article vjNeighbor-based representation
Step 5, according to the user uiLast interactive watchDisplay device
And user uiNeighbor-based representation
Calculating current pre-interaction item v by utilizing full connection layerjRepresentation of predictions
In an embodiment, according to user uiLast interactive representation
And user uiNeighbor-based representation
Calculating current pre-interaction item v by utilizing full connection layerjRepresentation of predictions
The calculation process is shown in formula (13):
wherein, W1And W2Is the parameter matrix of the fully-connected layer, and b is the bias of the fully-connected layer.
Step 6, according to the user uiAnd an article vjLast interactive representation
And
time interval between last interaction and current interaction
And
and neighbor-based representation
And
respectively calculating user u by utilizing two recurrent neural network layersiAnd an article vjCurrently interacted with representation
And
as shown in FIG. 2, two recurrent neural network layers RNNs are utilizedUAnd RNNVCalculate user u separatelyiAnd an article vjCurrently interacted with representation
And
RNNUis input by user uiLast interactive representation
Article vjLast interactive representation
User uiNeighbor-based representation
And the time interval between the last interaction and the current interaction of the user
RNNVIs an item vjLast interactive representation
User uiLast interactive representation
Article vjNeighbor-based representation
And the time interval between the last interaction and the current interaction of the object
RNNUAnd RNNVThe specific calculation formula of (2) is as follows:
wherein,
denotes RNNUThe network parameters of (a) are set,
denotes RNNVThe network parameters of (a) are set,
and
respectively time interval
And
the representation is derived by a fully connected layer, which is shared at different time intervals. Sharing RNN by all usersUTo update the user's representation, all items share the RNNVTo update the representation of the item. Will RNNUAnd RNNVAs a representation of the user and the item, respectively.
Step 7, according to the current pre-interaction object vjRepresentation of predictions
And a real representation
Error between, user uiRegularization loss and article vjRegularization loss, calculating the overall loss
Article vjThe representation after the last interaction is taken as the real representation before the current interaction
Minimizing items vjRepresentation of predictions
And a real representation
Mean square error between them to obtain the predicted loss, the overall loss
The calculation is as follows:
wherein the first term is the prediction loss and the last two terms are regularization terms to avoid representation of users and itemsToo large a change, λUAnd λIIs a scale parameter, | |)2Indicating the L2 distance.
Step 8, according to the loss of all samples in the batch
Network parameters in the entire model are adjusted.
Calculate the loss of all samples in the batch
The specific calculation method is as follows:
wherein
For each sample loss, M is the number of samples in the batch. In the present invention, according to the loss
Network parameters in the entire model are adjusted.
And 9, repeating the steps 2-8 until all batches of the training data set participate in model training.
Step 10, if the specified training iteration times are reached, the training is finished; otherwise, returning to the step 2.
And 11, predicting the articles which are possibly interacted by the user by using the temporal interaction network prediction model after parameter tuning.
Based on the user and item representation obtained after the training is finished, the user u is usediFor example, given user uiLast interactive representation
And user uiNeighbor-based representation
Computing representations of interactions involving item predictions
The specific process is shown in formula (13). Computing representations of item predictions
Representation of all objects
The distance L2 between them, the top-K items with the small distance L2 are the items that the user may interact with.
The above-mentioned embodiments are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions, equivalents, etc. made within the scope of the principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A multi-relation-aware temporal interaction network prediction method is characterized by comprising the following steps:
(1) with user uiAnd an article vjInteraction (u) occurring at time ti,vjT) constructing a training data set as a sample, and batching the training data set;
(2) for interaction (u)i,vjT), mining the nodes with historical interaction relationship, common interaction relationship and interaction sequence similarity relationship between the nodes based on historical interaction information to obtain interactive nodes uiAnd vjConstructing a local relationship graph before current interaction
And
(3) according to a local relationship diagram
And
obtaining user u through hierarchical multi-relation perception aggregationiNeighbor-based representation
And an article vjNeighbor-based representation
(4) According to user uiLast interactive representation
And user uiNeighbor-based representation
Calculating current pre-interaction item v by utilizing full connection layerjRepresentation of predictions
(5) According to user uiAnd an article vjLast interactive representation
And
time interval between last interaction and current interaction
And
and neighbor-based representation
And
respectively calculating user u by utilizing two recurrent neural network layersiAnd an article vjCurrently interacted with representation
And
(6) according to the current pre-interaction item vjRepresentation of predictions
And a real representation
Error between, user uiRegularization loss and article vjRegularization loss, calculating the overall loss
According to the loss of all samples in the batch
Adjusting network parameters in a temporal interaction network prediction model until all batches participate in model training, wherein the temporal interaction network prediction model comprises all full connection layers and a cyclic neural network layer used in the steps (2) to (6);
(7) and predicting the articles which are possibly interacted by the user by using the temporal interaction network prediction model after the parameters are adjusted.
2. The method of claim 1, wherein the training data set is batched using a t-n-Batch algorithm.
3. The method for predicting a temporal interaction network with multi-relationship awareness as claimed in claim 1, wherein the specific process of the step (2) is as follows:
local relationship diagram
Wherein
And
respectively represent and node niRelated node set, edge set, relation type set and relation attribute set, edge e is defined as triple
Representing a node niAnd node njThere is a relationship between them, the relationship type is
Comprises three types of historical interactive relationship, common interactive relationship and interactive sequence similarity relationship, and the relationship attribute is
Wherein q is (t, w), t represents a time attribute, and w represents a weight attribute;
the specific method of multi-relation derivation is as follows:
1) historical interaction relationships
If two nodes are interacted historically, a historical interaction relationship exists between the two nodes, the time attribute t of the historical interaction relationship is the last interaction time of the two nodes, and the weight attribute w is the historical interaction frequency;
2) mutual interaction relation
If two nodes interact with the same node in the T time period, a common interaction relationship exists between the two nodes. The time attribute t of the common interaction relationship is the time of the last common interaction of the two nodes, wherein the time of the common interaction is the closest time to the current time in the time of the interaction of the two nodes and the same node, and the weight attribute w is the historical common interaction times;
3) interaction sequence similarity relationship
All the interactive sequences are regarded as 'documents', each interactive sequence is regarded as 'sentences', nodes in the interactive sequences are regarded as 'words', and after the user interactive sequences and the article interactive sequences are respectively embedded by using a Doc2Vec model, the representation of each user based on the interactive sequences and the representation of each article based on the interactive sequences can be obtained;
as the interaction between the user and the article continuously occurs, the Doc2Vec model is updated in an incremental training mode, and a new representation of the user and the article based on the interaction sequence is obtained. Given two nodes n of the same typeiAnd njPresentation based on interaction sequences
And
calculating the cosine similarity between the two, wherein the calculation mode is as follows:
wherein, represents the dot product;
setting a threshold value mu only when the cosine similarity
And when the value is larger than the threshold value mu, the interaction sequence similarity relation exists between the two nodes. Interactive sequence phaseThe time attribute t of the similarity relation is the moment of interaction which occurs at last in the interaction sequence of the two nodes, and the weight attribute w is cosine similarity;
after the nodes with historical interaction relationship, common interaction relationship and interaction sequence similarity relationship with the interaction nodes are mined by the multi-relationship derivation concrete method, the interaction nodes u can be regarded as interaction nodes uiAnd vjConstructing a local relationship graph before current interaction
And
4. the method for predicting a temporal interaction network with multi-relationship awareness as claimed in claim 1, wherein the specific process of the step (3) is as follows:
if node niAs a user, the corresponding user is ujThen the last interactive representation of the node is
If node niIs an article, corresponding to the article vjThen the last interactive representation of the node is
Node niThe representation after the last interaction is recorded as
When node niWhen interaction occurs, the node is given a local relationship graph in the time interval between the last interaction and the current interaction
The interaction influence propagated by the neighbor node in which the interaction occurs, namely the representation of the neighbor node after the interaction occurs
Wherein M is the number of nodes interacted among the neighbor nodes, and the specific process of hierarchical multi-relationship perception aggregation is as follows:
the first layer is intra-relationship aggregation, neighbor nodes are aggregated according to the interaction influence transmitted by the same relationship type, corresponding weights are given to different neighbor nodes, and the neighbor representation of the node based on the specific relationship type is obtained, wherein the process is as follows:
for a given node niIs a neighbor node njThe input of the multi-head attention mechanism is
Then the input of attention mechanism of the kth head
The calculation is as follows:
wherein,
representing a matrix of k-th head input parameters, different relation types
The same is true. According to the input of the neighbor node
The attention coefficient for the kth head is calculated as follows:
wherein,
attention weight matrix representing kth head, different relation types
Instead, T represents the matrix transpose, | represents the vector join operation,
the weight associated with the relationship attribute q is represented and the calculation process is shown in equation (4).
Inputting the relation attribute q ═ t, w into the full-link layer to obtain an output value
If the relationship type r is a history interactive relationship, the t attribute represents the node niAnd a neighbor node njAt the last interaction time, the w attribute represents the historical interaction times of the two nodes; if the relationship type r is a common interaction relationship, the t attribute is the time of the last common interaction of the two nodes, and the w attribute is the historical common interaction times; if the relationship type r is an interaction sequence similarity relationship, the t attribute is the moment of interaction occurring at the last in the interaction sequences of the two nodes, and the w attribute is cosine similarity. The calculation formula is as follows:
wherein WfeatParameter matrix being a fully connected layer, bfeatFor the biasing of the fully connected layer, different heads of different relation types share the fully connected layer;
for a given node niAll the neighbor nodes with the relation types of the historical interaction relation are subjected to weight normalization to obtain neighbor nodes njNormalized kth head attention coefficient:
wherein,
is given node niThe relationship type is a neighbor node set of historical interaction relationship. Based on the above calculation, the hidden vector of the k-th head
The calculation is as follows:
for a given node niImplicit vectors obtained by K heads
Obtaining a node n after averagingiNeighbor representation based on historical interaction relationships
The calculation process is shown in formula (7):
the related parameters with the relationship type of the historical interactive relationship are converted into the related parameters of the common interactive relationship, and the neighbor expression based on the common interactive relationship is obtained by using the formulas (2) to (7)
Similarly, the related parameters with the relationship type of the historical interaction relationship are converted into the related parameters of the interaction sequence similarity relationship, and the neighbor expression based on the interaction sequence similarity relationship is obtained by using the formulas (2) to (7)
The related parameters comprise a relationship attribute q, a neighbor node set of each relationship type and an attention weight matrix
The second layer is inter-relationship aggregation, because the importance of the interaction influence propagated according to different relationship types to a given node is different, corresponding weights are given to the different relationship types by using a self-attention mechanism, and the specific process is as follows:
for node niNeighbor representation based on historical interaction relationships
Neighbor representation based on common interaction relationships
And neighbor representation based on inter-sequence similarity
Obtaining input of self-attention mechanism after splicing
The calculation process of the query matrix Q, the key matrix K and the value matrix V of the self-attention mechanism is as follows:
Q＝HWQ (8)
K＝HWK (9)
V＝HWV (10)
wherein
And
respectively, an inquiry weight matrix, a key weight matrix and a value weight matrix;output of self-attention mechanism
As shown in formula (11):
wherein,
is a scale factor, dk＝dv；
Inputting the output Z of the attention mechanism into the full-connection layer to obtain a node niNeighbor-based representation
The calculation process is shown in formula (12):
wherein, WoutParameter matrix being a fully connected layer, boutA bias for a fully connected layer;
according to a local relationship diagram
And
user u is obtained by hierarchical multi-relation perception aggregationiNeighbor-based representation
And an article vjNeighbor-based representation
5. The method for predicting a temporal interaction network with multi-relationship awareness as claimed in claim 1, wherein the specific process of the step (4) is as follows:
according to user uiLast interactive representation
And user uiNeighbor-based representation
Calculating current pre-interaction item v by utilizing full connection layerjRepresentation of predictions
The calculation process is shown in formula (13):
wherein, W1And W2Is the parameter matrix of the fully-connected layer, and b is the bias of the fully-connected layer.
6. The method for predicting a temporal interaction network with multi-relationship awareness as claimed in claim 1, wherein the specific process of the step (5) is as follows:
using two recurrent neural network layers RNNUAnd RNNVCalculate user u separatelyiAnd an article vjCurrently interacted with representation
And
RNNUis input by user uiLast interactive representation
Article vjLast interactive representation
User uiNeighbor-based representation
And the time interval between the last interaction and the current interaction of the user
RNNVIs an item vjLast interactive representation
User uiLast interactive representation
Article vjNeighbor-based representation
And the time interval between the last interaction and the current interaction of the object
RNNUAnd RNNVThe specific calculation formula of (2) is as follows:
wherein,
denotes RNNUThe network parameters of (a) are set,
denotes RNNVThe network parameters of (a) are set,
and
respectively time interval
And
by means of the representation obtained by the full connection layer, the full connection layer is shared by different time intervals, and RNN is shared by all usersUTo update the user's representation, all items share the RNNVTo update the representation of the item, the RNNUAnd RNNVAs a representation of the user and the item, respectively.
7. The method of claim 1, wherein in step (6), the overall loss is reduced
The calculation is as follows:
the first term is prediction loss, and the last two terms are regularization terms to avoid excessive representation change of users and articles, namely lambdaUAnd λIIs a scale parameter, | |)2Indicating the L2 distance.
8. The method for predicting a temporal interaction network with multi-relationship awareness as claimed in claim 1, wherein the specific process of the step (7) is as follows:
given user uiLast interactive representation
And user uiNeighbor-based representation
Computing representations of interactions involving item predictions
Then, a representation of the item forecast is computed
Representation of all objects
The distance L2 between them, the top-K items with the small distance L2 are the items that the user may interact with.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010797094.5A CN111949892B (en) | 2020-08-10 | 2020-08-10 | Multi-relation perception temporal interaction network prediction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010797094.5A CN111949892B (en) | 2020-08-10 | 2020-08-10 | Multi-relation perception temporal interaction network prediction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111949892A true CN111949892A (en) | 2020-11-17 |
| CN111949892B CN111949892B (en) | 2022-04-05 |
Family
ID=73331993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010797094.5A Active CN111949892B (en) | 2020-08-10 | 2020-08-10 | Multi-relation perception temporal interaction network prediction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111949892B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541129A (en) * | 2020-12-06 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Method and device for processing interaction event |
| CN113191400A (en) * | 2021-04-14 | 2021-07-30 | 中国海洋大学 | Method and device for retrieving corresponding three-dimensional model based on two-dimensional image |
| CN113537613A (en) * | 2021-07-28 | 2021-10-22 | 浙江大学 | Method for predicting temporal network by sensing motif |
| CN114969540A (en) * | 2022-06-10 | 2022-08-30 | 重庆大学 | Method for predicting future interaction behavior of user in social network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140362091A1 (en) * | 2013-06-07 | 2014-12-11 | Ecole Polytechnique Federale De Lausanne | Online modeling for real-time facial animation |
| CN110555469A (en) * | 2019-08-15 | 2019-12-10 | 阿里巴巴集团控股有限公司 | Method and device for processing interactive sequence data |
| US10650432B1 (en) * | 2016-11-28 | 2020-05-12 | Amazon Technologies, Inc. | Recommendation system using improved neural network |
| CN111159425A (en) * | 2019-12-30 | 2020-05-15 | 浙江大学 | Temporal knowledge graph representation method based on historical relationship and double-graph convolution network |
-
2020
- 2020-08-10 CN CN202010797094.5A patent/CN111949892B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140362091A1 (en) * | 2013-06-07 | 2014-12-11 | Ecole Polytechnique Federale De Lausanne | Online modeling for real-time facial animation |
| US10650432B1 (en) * | 2016-11-28 | 2020-05-12 | Amazon Technologies, Inc. | Recommendation system using improved neural network |
| CN110555469A (en) * | 2019-08-15 | 2019-12-10 | 阿里巴巴集团控股有限公司 | Method and device for processing interactive sequence data |
| CN111159425A (en) * | 2019-12-30 | 2020-05-15 | 浙江大学 | Temporal knowledge graph representation method based on historical relationship and double-graph convolution network |
Non-Patent Citations (1)
| Title |
|---|
| 刘辉等: "融合邻居选择策略和信任关系的兴趣点推荐", 《计算机工程与科学》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541129A (en) * | 2020-12-06 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Method and device for processing interaction event |
| CN112541129B (en) * | 2020-12-06 | 2023-05-23 | 支付宝(杭州)信息技术有限公司 | Method and device for processing interaction event |
| CN113191400A (en) * | 2021-04-14 | 2021-07-30 | 中国海洋大学 | Method and device for retrieving corresponding three-dimensional model based on two-dimensional image |
| CN113537613A (en) * | 2021-07-28 | 2021-10-22 | 浙江大学 | Method for predicting temporal network by sensing motif |
| CN113537613B (en) * | 2021-07-28 | 2023-11-21 | 浙江大学 | Temporal network prediction method for die body perception |
| CN114969540A (en) * | 2022-06-10 | 2022-08-30 | 重庆大学 | Method for predicting future interaction behavior of user in social network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111949892B (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111949892B (en) | Multi-relation perception temporal interaction network prediction method | |
| CN112364976A (en) | User preference prediction method based on session recommendation system | |
| CN112883289B (en) | PMF recommendation method based on social trust and tag semantic similarity | |
| CN115062732B (en) | Resource sharing cooperation recommendation method and system based on big data user tag information | |
| Wang | A survey of online advertising click-through rate prediction models | |
| CN113420221B (en) | Interpretable recommendation method integrating implicit article preference and explicit feature preference of user | |
| CN114840745B (en) | Personalized recommendation method and system based on graph sign learning and deep semantic matching model | |
| CN113850616A (en) | Customer life cycle value prediction method based on depth map neural network | |
| Pan et al. | Intelligent recommendation method integrating knowledge graph and Bayesian network | |
| CN117635238A (en) | Commodity recommendation method, device, equipment and storage medium | |
| Yin et al. | An efficient recommendation algorithm based on heterogeneous information network | |
| CN117056609A (en) | Session recommendation method based on multi-layer aggregation enhanced contrast learning | |
| CN116955810A (en) | Optimization method of knowledge collaborative recommendation algorithm based on graph convolution network | |
| Zhou et al. | Online recommendation based on incremental-input self-organizing map | |
| Li et al. | PRIMA++: A probabilistic framework for user choice modelling with small data | |
| CN115982373A (en) | Knowledge graph recommendation method combining multi-level interactive contrast learning | |
| CN115687757A (en) | Recommendation method fusing hierarchical attention and feature interaction and application system thereof | |
| CN115545833A (en) | Recommendation method and system based on user social information | |
| Jiang et al. | An Application of SVD++ Method in Collaborative Filtering | |
| Mitrevski et al. | Momentum-based gradient methods in multi-objective recommendation | |
| Murthy et al. | Model for Predicting Prospective Big-Mart Sales Based on Grid Search Optimization (GSO) | |
| CN110956528B (en) | Recommendation method and system for e-commerce platform | |
| Yang et al. | An algorithm for ordinal classification based on pairwise comparison | |
| CN113569139A (en) | Personalized session recommendation method and system | |
| Kang | Collaborative filtering algorithm based on trust and information entropy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |