CN114117229A - An Item Recommendation Method Based on Directed and Undirected Structural Information of Graph Neural Networks - Google Patents

An Item Recommendation Method Based on Directed and Undirected Structural Information of Graph Neural Networks Download PDF

Info

Publication number
CN114117229A
CN114117229A CN202111447363.6A CN202111447363A CN114117229A CN 114117229 A CN114117229 A CN 114117229A CN 202111447363 A CN202111447363 A CN 202111447363A CN 114117229 A CN114117229 A CN 114117229A
Authority
CN
China
Prior art keywords
vector
item
session
graph
items
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111447363.6A
Other languages
Chinese (zh)
Other versions
CN114117229B (en
Inventor
王庆梅
王铮
胡承佐
靳博文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology Beijing USTB
Original Assignee
University of Science and Technology Beijing USTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology Beijing USTB filed Critical University of Science and Technology Beijing USTB
Priority to CN202111447363.6A priority Critical patent/CN114117229B/en
Publication of CN114117229A publication Critical patent/CN114117229A/en
Application granted granted Critical
Publication of CN114117229B publication Critical patent/CN114117229B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明公开了一种基于有向和无向结构信息的图神经网络的项目推荐方法，包括：根据会话序列图的邻接关系，利用图卷积网络提取图中的无向结构信息，利用门控图神经网络提取图中的有向结构信息，再计算得到中间项目隐含向量，对得到的中间项目隐含向量通过线性变换得到最终的项目隐含向量；给会话序列中出现的重复点击项目分配更高的注意力，并在生成项目隐含向量的时候引入注意力机制，根据项目间依赖的程度修改相应项目的权重系数。本发明使得生成的会话向量在推荐过程中预测得更准确。

Figure 202111447363

The invention discloses a graph neural network project recommendation method based on directed and undirected structural information, comprising: extracting undirected structural information in the graph by using a graph convolutional network according to the adjacency relationship of the session sequence graph, and using gate control The graph neural network extracts the directed structure information in the graph, and then calculates the implicit vector of the intermediate item, and obtains the final item implicit vector through linear transformation on the obtained implicit vector of the intermediate item; assigns the repeated click items that appear in the session sequence Higher attention, and the attention mechanism is introduced when generating the item latent vector, and the weight coefficient of the corresponding item is modified according to the degree of inter-item dependency. The present invention enables the generated session vector to be more accurately predicted in the recommendation process.

Figure 202111447363

Description

Project recommendation method of graph neural network based on directed and undirected structural information
Technical Field
The invention relates to the technical field of recommendation methods, in particular to a project recommendation method of a graph neural network based on directed and undirected structural information.
Background
Recommendation systems are one of the most important downstream applications in the field of data mining and machine learning. It can help platform users alleviate the problem of information overload and sort out valuable information in many web applications for e-commerce platforms, music websites, and the like. In most recommendation systems the sequence of user actions is time-ordered and characterized by anonymity and large data volumes. In order to predict the behavior information of the user at the next moment, the recommendation based on the conversation sequence learns the preference of the user by mining the sequence order characteristic information in the historical behavior of the user.
A conversation sequence refers to a sequence of items generated by user clicks over a time interval; and the recommendation based on the conversation sequence can capture the importance of the dependency relationship in the sequence to the sequence prediction. In other words, users often have a common purpose in a certain sequence of sessions, such as purchasing summer clothing; the behavior characteristics of the user between different sequences may not be relevant, for example, the user may aim to purchase a cell phone accessory in other sessions. The recommendation based on the session sequence is to predict the next click of the user, i.e. the sequence tag v in the session sn+1. Using a recommendation model based on the sequence of conversations, the probability of all possible items is available for each conversation s
Figure BDA0003385139970000011
Wherein
Figure BDA0003385139970000012
Probability vector
Figure BDA0003385139970000013
All possible cases of the next click item occurring after the current session are included, and the value of each element represents the recommendation score of the corresponding item,
Figure BDA0003385139970000021
middle rowThe top K items are candidate items to be recommended.
In view of its high practical value, recommendations based on conversational sequences have received great attention in recent years and many well-worked research results have emerged. Early methods were based primarily on markov chains and recurrent neural networks. With the recent rise of neural networks and their unsophisticated performance in many downstream tasks, some research efforts have applied GNNs to conversational-based sequence recommendations. Although these GNNs-based methods perform well, there are some problems with these methods.
(1) Repeatedly appearing items in the click sequence are ignored. In fact, items that appear multiple times are not as important as other items, and to some extent, can reflect user preference information.
(2) The structural information in the session sequence diagram is not well utilized in generating the vector representation of the item. It is not sufficient to actually consider only the directionality between items, and introducing undirected relationships between items enables better learning of the user's behavioral information.
For example, the paper improvement conversation in session-based collaborative filtering by using temporal context of FONSECA et al proposes to convert the sparse session vector into a dense vector by using a clustering method; the Session-based formatting for predicting the next song of park.s.e et al, proposes a method of converting Session sequences into vectors and then calculating cosine similarity between the Session vectors. It can be seen that the neighborhood-based approach is simple but effective; meanwhile, the method is also influenced by data sparsity, and more importantly, the method does not consider the complex conversion relation among the items in the session vector.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a project recommendation method of a neural network based on directed and undirected structure information, so as to solve the problems of neglecting repeated projects in a click sequence and not well utilizing the structure information in a conversation sequence diagram when generating vector representation of the projects in the prior art.
In order to solve the above technical problem, the present invention provides a project recommendation method based on a neural network of a graph with directed and undirected structural information, the overall framework of which is shown in fig. 1, and the method comprises the following steps:
s1, let V represent the collection of items appearing in all conversation sequences, i.e.
Figure BDA0003385139970000031
Then an anonymous session sequence of length n may be used
Figure BDA0003385139970000032
Is expressed and the items in the conversation s are chronologically arranged, each viE.g. V represents the item clicked by the user in the session s, and the recommendation is to predict the next click of the user, i.e. predict the sequence label V in the session sn+1；
S2, receiving the historical conversation sequence, and converting the historical conversation sequence into a directed conversation sequence chart Gs＝(Vs,εs,As) In which V issRepresents a set of points,. epsilonsRepresents the edge set, AsRepresents a set of adjacency matrices, AsDefined as three adjacency matrixes
Figure BDA0003385139970000033
And
Figure BDA0003385139970000034
the splicing of (a), wherein,
Figure BDA0003385139970000035
representing weighted adjacency matrices of undirected graphs, and
Figure BDA0003385139970000036
and
Figure BDA0003385139970000037
respectively representing weighted in-degree adjacent momentsArray and outgoing adjacency matrix;
s3, connecting point viMapping the e to the random embedded vector space to obtain d-dimensional vector representation
Figure BDA0003385139970000038
Extracting a first intermediate implicit vector of an item in the conversation sequence diagram by using a graph convolution network, extracting a second intermediate implicit vector of item conversion in the conversation sequence diagram by using a gated diagram neural network, and obtaining an item implicit vector through first linear transformation;
s4, inputting the item implicit vector into the target attention network, thereby obtaining a session implicit vector corresponding to the session sequence S;
s5, acquiring global information and local information of the conversation sequence S, and constructing a conversation vector representation through second linear transformation;
s6, predicting the probability of all possible items being clicked on the conversation sequence S by the softmax function, and recommending items with a high probability.
Further, a first intermediate implicit vector of an item in the session sequence diagram is extracted by using the graph volume network, that is, undirected structure information with attention is extracted by using the graph volume network, and the steps are as follows:
s31, generating a feature matrix X of the conversation sequence diagram; each node v in the session sequence diagramiCorresponding d-dimensional feature vector
Figure BDA0003385139970000041
The stack of (2) constituting a feature matrix of a session sequence diagram
Figure BDA0003385139970000042
Figure BDA0003385139970000043
X＝[x1,…,xn]T；
S32, for the graph convolution layer of the k layer, use the matrix H(k-1)Input vectors representing all nodes, by H(k)An output vector representing the node, wherein the initial d-dimensional node vector is the initial inputThe first layer of the graph convolution layer network is characterized by the following formula:
H(0)＝X, (1)
before the input of each graph convolution layer, each node viThe feature of (a) is averaged with the feature vector of its local neighbor, and the calculation formula is:
Figure BDA0003385139970000044
wherein, aijIs node viAnd vjEdge weight between, di＝∑jaij；
S33, the output of the graph convolution network is a first intermediate implicit vector.
Further, for equation (2), the simple matrix operation of the whole graph is simplified, and S represents the result after symmetric normalization, as follows:
Figure BDA0003385139970000051
Figure BDA0003385139970000052
Figure BDA0003385139970000053
wherein, for the formula (3),
Figure BDA0003385139970000054
and is
Figure BDA0003385139970000055
Is that
Figure BDA0003385139970000056
An degree matrix of,. is a point-by-point operator.
Figure BDA0003385139970000057
Weighted adjacency matrix that is an undirected graph. I is the identity matrix.
Further, in order to increase the weight of the edge item and reduce the interference of the noise of other items, the propagation matrix
Figure BDA0003385139970000058
The left half of equation (4) for the items with edges connected in (b) is weighted up, i.e., the attention of the self information in the matrix is raised, as shown in fig. 2.
Further, with respect to the formula (4), α and β are hyperparameters, respectively, to control the ratio of the propagation matrix information and the unit matrix information, thereby controlling the absorption ratio of the node information with attention during the propagation process. As shown in FIG. 2, the adjacency matrix
Figure BDA0003385139970000059
And propagation matrix
Figure BDA00033851399700000510
Item v with repeated clicks therein2And item v converted during repeat click3There will be higher attention information, i.e. weight.
The above steps carry out local smoothing on the implicit vector representation of the nodes along the graph, and after the graph convolution network is used as a feature preprocessing method to transmit the features, the nodes can absorb the attention information of adjacent nodes, and finally the locally connected nodes can have similar prediction performance;
further, a second intermediate implicit vector of item conversion in the session sequence diagram is extracted by using a gated graph neural network, namely, the directed structure information with attention is extracted by using the gated graph neural network, and the steps are as follows:
for nodes in the session sequence chart, the node vector updating steps are as follows:
Figure BDA00033851399700000511
Figure BDA0003385139970000061
Figure BDA0003385139970000062
Figure BDA0003385139970000063
Figure BDA0003385139970000064
wherein, for the formula (6),
Figure BDA0003385139970000065
and
Figure BDA0003385139970000066
the weights and the magnitude of the bias terms are controlled,
Figure BDA0003385139970000067
to represent the result of the interaction between a node and an adjacent node,
Figure BDA0003385139970000068
and
Figure BDA0003385139970000069
reset gate and update gate, respectively, weight matrix Wz、Uz，Wr、UrAnd Wo、UoRespectively represent learnable network parameters in the reset gate, the update gate and the output gate,
Figure BDA00033851399700000610
represents node viIs used to generate the second intermediate implicit vector of (c),
Figure BDA00033851399700000611
is a sequence of node vectors in the session, and
Figure BDA00033851399700000612
Figure BDA00033851399700000613
for the first intermediate implicit vector output by the graph convolution network, σ (·) is a sigmoid function, which is a point-by-point operator. Adjacency matrix
Figure BDA00033851399700000614
Representing the communication of the nodes in the graph,
Figure BDA00033851399700000615
representative node viIn that
Figure BDA00033851399700000616
Two columns of matrix blocks.
Wherein, the matrix
Figure BDA00033851399700000617
Is defined as an in-degree matrix
Figure BDA00033851399700000618
Sum degree matrix
Figure BDA00033851399700000619
Which represent weighted connections of the input and output edges, respectively, in the session sequence diagram. For example, given a session sequence s ═ v1,v2,v3,v2,v4]Corresponding session sequence diagram GsAnd adjacency matrix
Figure BDA00033851399700000620
As shown in fig. 3;
further, a project implicit vector is generated, which comprises the following steps:
GCN and GCN over graph convolution networkAfter the information of the gate control graph neural network GGNN is processed, a second intermediate implicit vector is obtained
Figure BDA00033851399700000621
In order to balance the proportion of the non-directional structural information with attention and the directional structural information, the following formula is adopted for control:
Figure BDA00033851399700000622
wherein gamma is a hyper-parameter, thus obtaining a final accurate item implicit vector H;
after the implicit vector representation of each item is obtained, a target vector is further constructed, so that the correlation of historical behaviors can be analyzed on the premise of considering the target item. The target items are all items to be predicted.
Computing all items v in a conversation s using a local target attention modeliFor each target item vtAn attention score β for e Vi,tWherein
Figure BDA0003385139970000071
And
Figure BDA0003385139970000072
are respectively item viAnd vtIs determined by the second intermediate implicit vector representation of (1).
Figure BDA0003385139970000073
In equation (12), all items in the conversation sequence are matched with the target item respectively, and the weighted matrix is used
Figure BDA0003385139970000074
To perform a pair-wise nonlinear transformation; the resulting self-attention score is then normalized by the softmax function to obtain the final attention score βi,t；
Finally, for each conversation sequence s, the user is directed to a target item vtCan be expressed as
Figure BDA0003385139970000075
I.e. vectors based on target attention
Figure BDA0003385139970000076
Figure BDA0003385139970000077
It represents the level of interest a user has in generating between different target items;
further, a conversation sequence vector is generated, which comprises the following steps:
representing a user's short-term interest as a local vector
Figure BDA0003385139970000078
By the last item in the conversation sequence
Figure BDA0003385139970000079
Represents the local vector, as shown in equation (14):
Figure BDA00033851399700000710
defining a user's long-term preferences as a global vector
Figure BDA00033851399700000711
Where all the appearing item vectors in session s are aggregated. While using a mechanism of attention to introduce items of last interaction
Figure BDA0003385139970000081
With items [ v ] appearing throughout the conversation1,v2,…,vn]The dependency relationship between them.
Figure BDA0003385139970000082
Figure BDA0003385139970000083
Wherein the ratio of q,
Figure BDA0003385139970000084
and W1,
Figure BDA0003385139970000085
Is a corresponding weight parameter, αiRepresenting the dependency between the last item and the items that appear in the entire sequence of conversations.
And finally, splicing the local vector, the global vector and the vector based on the target attention obtained in the previous step, and obtaining a session vector corresponding to the session sequence s by utilizing linear conversion.
Figure BDA0003385139970000086
Wherein the weight parameter
Figure BDA0003385139970000087
Projecting the results of three vector concatenations into vector space
Figure BDA0003385139970000088
Performing the following steps;
further, the step of generating the recommendation at step S6 is as follows:
all the items viSecond implicit vector of e.V
Figure BDA0003385139970000089
The session vector s corresponding theretohThe multiplication is carried out, and the result is obtained,
Figure BDA00033851399700000810
then obtaining an output vector of the model through a softmax function pair
Figure BDA00033851399700000811
Figure BDA00033851399700000812
Wherein
Figure BDA00033851399700000813
Represents the predicted recommendation scores of all the target items, and
Figure BDA00033851399700000814
representing the probability that the target item is clicked at the next moment in the session sequence s.
Figure BDA00033851399700000815
The top K items with the highest ranking are the items to be recommended.
For each session sequence diagram, defining a loss function as the cross entropy of the predicted value and the actual value,
Figure BDA00033851399700000816
wherein y isiA one-hot encoded vector representing the actual click item at the next instance in the session sequence.
Finally, the steps S2-S6 are iteratively trained by using a time-based back propagation algorithm BPTT algorithm to generate parameters related thereto, such as W, α, β, and the like.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic diagram of the overall framework of the model provided by the present invention;
FIG. 2 is an exemplary session sequence diagram and its corresponding undirected adjacency matrix with weights provided by the present invention
Figure BDA0003385139970000091
And weighted retransmission matrix
Figure BDA0003385139970000092
A schematic diagram;
FIG. 3 is an exemplary session sequence diagram and its corresponding adjacency matrix provided by the present invention
Figure BDA0003385139970000093
A schematic diagram;
FIG. 4 is a graph showing the behavior of different components in the P @20 index between structural information components in an ablation experiment according to the present invention;
FIG. 5 is a graph illustrating the behavior of the invention in terms of different components in the MRR @20 index between structural information components in an ablation experiment;
FIG. 6 is a graph of the present invention showing the behavior of different components in the P @20 index between the components of the attention information in the ablation experiment);
FIG. 7 is a graph illustrating the behavior of different components in the MRR @20 index of the component of the attention information in an ablation experiment according to the present invention;
FIG. 8 is a flowchart illustrating an item recommendation method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention provides a project recommendation method of a graph neural network based on undirected structure information and directed structure information, aiming at the problems that the existing method ignores projects which repeatedly appear in a click sequence, and the structure information in a conversation sequence graph is not well utilized when vector representation of the projects is generated. The method gives the prediction of the item clicked by the user at the next moment according to the current conversation sequence data of the user, and does not depend on the long-term preference information of the user in the process.
For the recommendation based on the conversation sequence, firstly, a directed conversation sequence diagram is constructed from the information of the historical conversation sequence; and extracting undirected structure information and directed structure information of item conversion in the conversation sequence diagram respectively by using a graph convolution network GCN and a gate control graph neural network GGNN, generating an accurate item implicit vector, inputting the obtained item implicit vector into an attention network, and considering global information and local information of the conversation simultaneously, thereby constructing a more reliable conversation representation and deducing a next click item.
As shown in fig. 1 and 8, the item recommendation method of the neural network based on undirected structure information and directed structure information includes:
s1, let V represent the collection of items appearing in all conversation sequences, i.e.
Figure BDA0003385139970000111
Then an anonymous session sequence of length n may be used
Figure BDA0003385139970000112
Is expressed and the items in the conversation s are chronologically arranged, each viE.g. V represents the item clicked by the user in the session s, and the recommendation is to predict the next click of the user, i.e. predict the sequence label V in the session sn+1。
S2, converting the historical conversation sequence into a directed conversation sequenceDrawing Gs＝(Vs,εs,As) In which V issRepresents a set of points,. epsilonsRepresents the edge set, AsRepresenting a set of adjacency matrices. In the session sequence chart GsEach node in the set represents an item viE.g. V, and each edge (V)i-1,vi)∈εsAll represent that the user successively clicks the item vi-1And item viAnd A issDefined as three adjacency matrixes
Figure BDA0003385139970000113
And
Figure BDA0003385139970000114
the splicing of the two pieces of the paper is carried out,
Figure BDA0003385139970000115
representing weighted adjacency matrices of undirected graphs, and
Figure BDA0003385139970000116
and
Figure BDA0003385139970000117
respectively representing weighted in-degree adjacency matrix and out-degree adjacency matrix.
S3, pair session sequence chart GsProcessing to obtain the item implicit vectors of all nodes in the session sequence diagram;
s4, inputting the item implicit vector into a target attention network, and obtaining a conversation sequence attention-based vector;
s5, acquiring global information and local information of the conversation sequence S, and constructing a conversation vector representation through second linear transformation;
s6, predicting the probability of all the target items being clicked on the conversation sequence S by the softmax function, and recommending items with a high probability.
In step S3, the session sequence chart GsEach node v iniMapping the e s to a random embedded vector space to obtain a d-dimensional vector representation
Figure BDA0003385139970000118
And obtaining an item implicit vector H through graph convolution network, gated graph neural network and linear processing.
The method specifically comprises the following steps:
(1) according to the session sequence chart GsGenerating each item viInitial item implicit vector x corresponding to e ViWherein V represents the set of items that appear in all sequences of conversations; each node v in the session sequence diagramiCorresponding d-dimensional feature vector
Figure BDA0003385139970000121
The stack of (2) constituting a feature matrix of a session sequence diagram
Figure BDA0003385139970000122
X＝[x1,…,xn]T。
Generating xiThe method comprises the following specific steps:
1) session sequence diagram GsWeighted undirected adjacency matrix in (1)
Figure BDA0003385139970000123
Is a sparse and symmetric adjacency matrix in which aijRepresents node viAnd vjThe edge weight between the nodes, and the no connection relation between the nodes is expressed as aij＝0。
2) Defining the degree matrix D as the diagonal matrix D ═ diag (D)1,…,dn) And the value on the diagonal is equal to the sum d of the row elements of the adjacent matrixi＝∑jaij。
3) By passing
Figure BDA0003385139970000124
Get each node v in the graphiAll have corresponding d-dimensional feature vectors
Figure BDA0003385139970000125
Figure BDA0003385139970000126
So the feature matrix of the conversation sequence
Figure BDA0003385139970000127
Namely, stacking of feature vectors corresponding to each node in the graph, i.e., X ═ X1,…,xn]T。
(2) X is to beiAnd inputting the graph convolution network with the undirected structure information to obtain a first intermediate implicit vector of all nodes in the graph (the first intermediate implicit vector has the undirected structure information).
(3) And obtaining a second intermediate implicit vector (with the directional structure information) of all nodes in the graph through the gated graph neural network with the directional structure information.
(4) And inputting the second intermediate implicit vector into the first linear transformation to obtain an accurate item implicit vector.
The graph neural network has natural adaptability to the recommendation based on the conversation sequence, because the graph neural network can automatically extract the characteristics of the conversation sequence graph under the premise of considering rich node connection relation.
Similar to convolutional neural networks CNNs, multi-layer perceptron MLPs, the graph convolution network GCN is for each node v in a multi-layer structureiThe features of (a) are learned and a new feature representation is obtained and then input into the corresponding linear classifier.
In the step (2), the step of generating the first intermediate implicit vector includes:
s31, for the graph convolution layer of the k layer, use the matrix H(k-1)Input vector h representing all nodesiBy H(k)An output vector representing the node. The initial d-dimensional node vector is the feature of the initial input and is input into the first-layer GCN:
H(0)＝X, (1)
a GCN with a number of layers K is equivalent to the eigenvector x for all nodes in the graphiA K-layer MLPs model is applied, except that the implicit vector representation of each node is averaged with its neighbor nodes at the beginning of each layer. Direction of nodes in each graph convolution layerThe volume representation has three update phases: feature propagation, linear transformation and point-by-point nonlinear activation. The only feature propagation stage used in the present invention for learning the item implicit vector.
S32, at the beginning of each layer, each node ShIs averaged with the feature vector of its local (random) neighbors:
Figure BDA0003385139970000131
the input of the graph convolution network is the first intermediate implicit vector.
Preferably, equation (2) is simplified with a simple matrix operation of the whole graph, as follows:
let S represent the contiguous matrix with self-loops after "symmetric normalization":
Figure BDA0003385139970000132
Figure BDA0003385139970000133
Figure BDA0003385139970000141
wherein
Figure BDA0003385139970000142
And is
Figure BDA0003385139970000143
Is that
Figure BDA0003385139970000144
The degree matrix of (n) is a dot-by-dot operator, and I is an identity matrix. Due to the adjacent matrix
Figure BDA0003385139970000145
Taking into account the weight of the repeated items, so the matrix
Figure BDA0003385139970000146
With additional attention paid to repeatedly clicking on the item. Meanwhile, due to the fact that S is provided with a self-loop, the symmetrical normalization process causes the weight of the items connected with multiple edges to be smaller than that of the items connected with a single edge or without the edges. Alpha, beta are hyper-parameters.
In order to increase the weight of the edge term and reduce the interference of the noise of other terms, the propagation matrix
Figure BDA0003385139970000147
For the items with connected edges, the weights are increased through the left half part of formula (4), i.e. the attention of self information in the matrix is increased.
The hyper-parameter alpha and the hyper-parameter beta are used for controlling the proportion of the propagation matrix information and the unit matrix information, thereby controlling the absorption proportion of the node information with attention in the propagation process. The adjacency matrix can be seen from the specific example given in fig. 2
Figure BDA0003385139970000148
And propagation matrix
Figure BDA0003385139970000149
Item α with repeat clicks therein2And item alpha converted during repeat clicks3There will be higher attention information, i.e. weight.
Thus, the equivalent updated form of equation (2) can be changed to a simple sparse matrix multiplication for all nodes.
The above steps locally smooth the implicit vector representation of the nodes along the graph, and after the graph convolution network is used as a feature preprocessing method to transmit the features, the nodes can absorb the attention information of adjacent nodes, and finally the locally connected nodes can have similar prediction performance.
In step (3), a gated graph neural network GGNN is constructed using the method of Li et al.
For session sequence diagram GsNode v iniThe node vector updating steps are as follows:
Figure BDA0003385139970000151
Figure BDA0003385139970000152
Figure BDA0003385139970000153
Figure BDA0003385139970000154
Figure BDA0003385139970000155
in equation (6), the adjacency matrix
Figure BDA0003385139970000156
Representing the communication of the nodes in the graph,
Figure BDA0003385139970000157
representative node viIn that
Figure BDA0003385139970000158
Two columns of matrix blocks in (1) are,
Figure BDA0003385139970000159
is a sequence of node vectors in the session, and
Figure BDA00033851399700001510
namely, the final output of the graph convolution network is used as the initial input of the gated graph neural network,
Figure BDA00033851399700001511
and
Figure BDA00033851399700001512
the weights and the magnitude of the bias terms are controlled,
Figure BDA00033851399700001513
which is used to represent the result of the interaction between a node and an adjacent node.
For equation (7), the reset gates are obtained by sigma (·) sigmoid function
Figure BDA00033851399700001514
And a retrofit gate
Figure BDA00033851399700001515
Weight matrix Wz、Uz，Wr、UrAnd Wo、UoNetwork parameters that can be learned in the reset gate, the update gate and the output gate, respectively, are point-by-point operators. Finally, the
Figure BDA00033851399700001516
Represents node viImplicit vectors generated by GGNN gated graph neural networks.
Matrix array
Figure BDA00033851399700001517
Is defined as an in-degree matrix
Figure BDA00033851399700001518
Sum degree matrix
Figure BDA00033851399700001519
Which represent weighted connections of the input and output edges, respectively, in the session sequence diagram. For example, given a session sequence s ═ v1,α2,v3,v2,v4]Corresponding session sequence diagram GsAnd adjacency matrix
Figure BDA00033851399700001520
As shown in fig. 3. It can be seen that the weights in the directed adjacency matrix are set according to the degree of closeness between nodes, e.g., v2From alpha3To alpha4Each having an edge, but the two weights are different because of α2And alpha3The more edges are connected with each other, which means that the similarity between the two is higher. To achieve better prediction effect, from alpha2Should absorb a more3So the model should pay more attention to v3On the body other than v4。
Therefore, the sequence diagram G is for each sessionsThe GGNN model propagates node information with attention between adjacent nodes, while reset gating and update gating determine the next information that needs to be discarded or retained, respectively.
In the step (4), after the information processing of the graph convolution network GCN and the gated graph neural network GGNN, the information is respectively obtained
Figure BDA0003385139970000161
The former is to perform attention-bearing undirected structure information processing on an initial embedded vector, and the latter is to extract attention-bearing directed structure information in a graph structure more finely on the basis of the former.
In order to balance the proportion of the non-directional structural information with attention and the directional structural information, the following formula is adopted for control:
Figure BDA0003385139970000162
wherein gamma is a hyperparameter, thus obtaining a final precise item implicit vector H.
In step S4, the specific steps include:
s41, calculating all items v in the conversation S by using a local target attention model (the model is prior art and is not described in detail)iFor each target item vtAn attention score β for e Vi,tWherein
Figure BDA0003385139970000163
And
Figure BDA0003385139970000164
are respectively item viAnd vtIs represented by an implicit vector.
Figure BDA0003385139970000165
In the above equation, items in the conversation are matched with the candidate targets, respectively, and the weighted matrix is used
Figure BDA0003385139970000166
To perform a pair-wise nonlinear conversion. The resulting self-attention score is then normalized by the softmax function and the final attention score is obtained.
S42, for each conversation sequence S, the user aims at the target item vtCan be expressed as
Figure BDA0003385139970000167
Figure BDA0003385139970000168
Finally, a vector based on the target attention is obtained
Figure BDA0003385139970000171
Which represents the level of interest a user has in generating between different target items.
In step S5, the short-term and long-term preferences of the user are further explored using the item vectors involved in the session S, so as to obtain local vectors and global vectors in the session, and a final session vector is generated by synthesizing the target attention-based vectors calculated in the above section.
And S51, acquiring a local vector. In a session sequence s, withThe final behavior of the user is often determined by the last interactive item in the current sequence. Therefore, the short-term interest of the user is expressed as a local vector
Figure BDA0003385139970000172
And the local vector is the last item in the conversation sequence
Figure BDA0003385139970000173
Is represented by a vector of (a).
Figure BDA0003385139970000174
S52, obtaining a global vector, and defining the long-term preference of the user as the global vector
Figure BDA0003385139970000175
Figure BDA0003385139970000176
Where all the appearing item vectors in session s are aggregated. While also taking advantage of the attention mechanism to introduce items of last interaction
Figure BDA0003385139970000177
With items [ v ] appearing throughout the conversation1,v2,…,vn]The dependency relationship between them.
Figure BDA0003385139970000178
Figure BDA0003385139970000179
Wherein the ratio of q,
Figure BDA00033851399700001710
and W1,
Figure BDA00033851399700001711
Are the corresponding weight parameters.
And S53, splicing the obtained local variable, the global variable and the target attention-based vector, and obtaining a session vector corresponding to the session sequence S by utilizing linear conversion.
Figure BDA00033851399700001712
Wherein the weight parameter
Figure BDA00033851399700001713
Projecting the results of three vector concatenations into vector space
Figure BDA00033851399700001714
In (1). It is noted that different session vectors may be generated for different target items (items in the session sequence) correspondingly.
In step S6, a session vector S corresponding to each session sequence S is obtainedhThereafter, for all items viScore for e.V
Figure BDA0003385139970000181
Performing calculations, i.e. vectors of candidate items
Figure BDA0003385139970000182
And session vector shThe multiplication is carried out, and the result is obtained,
Figure BDA0003385139970000183
then obtaining an output vector of the model through a softmax function pair
Figure BDA0003385139970000184
Figure BDA0003385139970000185
Wherein
Figure BDA0003385139970000186
Represents the predicted recommendation scores of all the target items, and
Figure BDA0003385139970000187
representing the probability that the target item is clicked at the next moment in the session sequence s,
Figure BDA0003385139970000188
the top K items with the highest ranking are the items to be recommended.
Sequence diagram G for each sessionsDefining the loss function as the cross entropy of the predicted value and the actual value,
Figure BDA0003385139970000189
wherein y isiA one-hot encoding vector (one-hot embedding) representing the real click item at the next instant in the session sequence.
In use, the data sets may be used to iteratively train steps S2-S6, such as training using the time-based back propagation BPTT algorithm, to obtain parameters such as W, W1, W2, W3, etc. in the above steps, which may be initially randomly set and then learned during training.
In training, each sequence is used as a training sample, so the total error is the sum of the errors at each time step (recommendation). Note that in a recommendation scenario based on conversational sequences, most conversations are relatively short sequences. To prevent the occurrence of overfitting, a smaller number of training passes is used.
Experimental analysis:
1. experimental data set
The method was evaluated in actual practice using the public data set Digineica published as the public data set Yoochoose and CIKM Cup 2016. The Yoochoose data set contains the user click stream within 6 months on the electronic shopping platform, and the Diginetica data set only contains the data of successful transaction, namely the purchase stream of the user.
At the same time, corresponding sequences and tags are further generated by slicing the input sequence data. For an input session sequence s ═ v1,v2,…,vn]As a data enhancement strategy, a series of sequences and tags ([ v ] are generated1],v2)，([v1,v2],v3)，…，([v1,v2,…,vn-1],vn) Among them, [ v ]1,v2,…,vn-1]Is the sequence generated, and vnA label representing the item, i.e. the sequence, clicked on at the next moment.
The details of the data set finally used are shown in table 1.
TABLE 1 Experimental data set statistics
Figure BDA0003385139970000191
2. Evaluation criteria
After the data set is determined, two metrics that are very common in the recommendation based on the conversation sequence are adopted as evaluation indexes of the algorithm.
(1) P @20(Precision) is a widely used measure of prediction accuracy. It represents the proportion of correct recommendations in the top 20 items of the algorithm recommendation.
(2) MRR @20(Mean Recircular Rank) is the Reciprocal Rank Mean of the correctly recommended items in the algorithm recommendations. When the true result exceeds 20 in the recommended ranks of the algorithm, the corresponding reciprocal rank is 0. The MMR metric is a method considering the recommendation order, and a larger MRR value represents that the real result is positioned at the top of the ranking list in the recommendation list, so that the effectiveness of the recommendation system is proved.
3. Experimental setup
The dimension of the implicit vector is set to d 100 in both datasets. All hyper-parameter settings utilize a mean of 0, standard deviationInitialization is performed for a gaussian distribution function of 0.1. These parameters were also optimized using a small batch Adam optimizer and the initial learning rate η was set to 0.001 and attenuated by 0.1 every three training cycles. Further, the batch size was set to 100, and the L2 regularization parameter was set to 10-5。
4. Analysis of Experimental results
The performance of several methods on both the P @20 and MRR @20 indices is shown in table 2, where bolding shows the best results. The method provided by the invention can flexibly construct the relation between the items on the conversation sequence diagram, and extract the directed structure information and the undirected structure information with attention, so that the subsequent learning of target attention can be more accurate, and the final recommendation can be given by integrating the global interest and the local interest of the user in the conversation. From the experimental data in table 2, it is clear that the method achieves the best performance results on both indexes on the three data sets, which proves the effectiveness of the method.
Conventional recommendation methods such as POP and S-POP do not perform well on the problem based on session sequences because they ignore the user' S preferences in the current session and only consider the top K most popular items. BPR-MF states that it is meaningful to use semantic information in a conversation, while better performing FPMC states that modeling conversation sequences using first-order Markov chains is a relatively efficient method. Also as a traditional recommendation method, Item-KNN is superior to the first two. It is worth noting that Item-KNN relies only on computing similarity between items, which suggests that the simultaneous presence of items is also a relatively important piece of information. However, Item-KNN does not take into account timing information in the conversation, and there is no way to capture information for switching between items.
Unlike the traditional method, the deep learning-based method basically has better performance in the index results of all data sets. GRU4Rec is a recurrent neural network-based method that can perform better than and to a similar degree as some conventional methods. This demonstrates that the recurrent neural network has some modeling capability for sequence data. However, GRU4Rec focuses mainly on modeling the session sequence and cannot capture the user preferences in the session. Later emerging methods such as NARM and STAMP both significantly improved GRU4 Rec. Nar explicitly captures the user's main preferences in the session, whereas STAMP takes advantage of the attention mechanism to consider the user's short-term interests, which is why they are superior to GRU4 Rec. The RepeatNet is also an algorithm based on a recurrent neural network, and achieves a better prediction effect by considering the repeated click behavior of the user, which indicates that the model has certain importance on the behavior habit of the user. However, repatnet has limited promotion over NARM and STAMP, possibly because modeling the user's repeated click habits through project features alone is inadequate and RNN-based structures cannot capture some common dependencies within a session.
The graph neural network based approach constructs each session sequence as a subgraph and encodes all the items in the session through the graph neural network. Both SR-GNN and TAGNN gave better results than all RNN-based models. SR-GNN utilizes a gated graph neural network to learn dependencies between items within a sequence of conversations, while TAGNN further exploits the dependencies between items within a conversation and target items with a mechanism of attention. However, these methods completely learn according to the directed relationship of the session sequence diagram, and do not comprehensively consider the undirected relationship in the session sequence diagram, because the relationship between the items in the session sequence is sometimes not unidirectional but bidirectional, and the more comprehensive relationship between the items can be captured by using undirected structure information. Moreover, they ignore the repetitive click feature that occurs in conversational sequences, and intuitively should the importance of a repeated occurrence of an item in a sequence be greater. In addition, in an actual recommendation scene, the association degree between items is variable, and the methods adopt an averaging method for the dependency relationship between items in the session, so that the different dependency degree of a certain item on other items cannot be reflected through a weighting or attention method.
The methods presented herein perform better than the methods described above. Specifically, there were relative lifts of 3.55%, 1.38%, 1.18% for the P @20 pair of best performing correlation methods and 1.92%, 4.34%, 1.98% for the MRR @20 pair of best performing correlation methods on the three datasets. The method can well extract the structural information in the session sequence diagram, sequentially extracts the undirected structural information and the directed structural information in the diagram by utilizing the graph convolution network and the gated graph neural network, and linearly combines the undirected structural information and the directed structural information so as to achieve accurate expression of the vector. In addition, repeated click items in the conversation sequence are considered, and the weight of repeated information is improved through an attention network; meanwhile, the self-information proportion of the nodes in the conversation sequence diagram is improved by adding self-loop and matrix operation, so that the nodes are not easily interfered by noise of other nodes. And then different weights are distributed by using the attention network according to different dependency relations among the projects, so that the network can generate accurate vector representation.
TABLE 2 comparison of the results
Figure BDA0003385139970000231
Ablation experiment:
the method can flexibly capture the relationship between the structural information and the items in the conversation sequence diagram. In order to verify the actual effect of each composition in the model, several model variants were set up for ablation experiments. In the experimental link, SR-GNN is selected as a reference method for comparison, and data in the experiment are displayed in the form of relative promotion percentage of comparison SR-GNN.
Firstly, performing combined analysis of directed structure information and undirected structure information: (a) -GCN, extracting only undirected structure information in the session sequence diagram. (b) GNN, extracting only the directed structure information in the session sequence graph. (c) GCN + GNN, inputting random initial vectors into two neural networks simultaneously, and then linearly combining the model output results. (d) -GCN + GNN (GCN), inputting the random initial vector into GCN, taking the output vector of GCN model as the input of GNN model, and finally linearly combining the output results of the two models. The results of the experimental comparison are shown in fig. 4 and 5.
Where AVG represents the average performance of the four combination conditions over the three data sets, respectively. As can be seen from fig. 4 and 5, the method GCN + Gnn (GCN) for integrating the directional structure information and the undirected structure information obtains the best results on both indicators P @20 and MRR @20 of the three data sets, which proves the importance of comprehensively considering the directional structure information and the undirected structure information. The average data AVG in fig. 5 also shows that the undirected structure information alone performs better than the directed structure information alone, whereas the directed structure information performs slightly better than the undirected structure information in terms of the performance of the single data set, on the MRR @20 index of the Yoochoose 1/4 data set and the dignetica data set instead. This reflects to some extent the connection between the user's preferences and the items in the conversation sequence based recommendations, the direction of the transition between items being of different importance in different scenarios, but on average also the absence of structural information is more important. This illustrates that, in the context of conversation-based sequence recommendation, the direction of the user's transition between items is worth considering, but the user needs to consider the relationship between the items viewed by the user in order to learn the user's preference better. And better than the former two methods, namely the directional structure information and the undirected structure information are comprehensively considered, the comprehensive performance and the average performance AVG of the GCN and the GNN in the methods of the figure 4 and the figure 5 are basically better than the performance of the GCN or the GNN which is singly used. The input data of the two network models in the GCN + GNN method are random embedded vectors, and the input of the GNN model in the GCN + GNN (GCN) method is a vector for extracting the undirected structure information through the GCN model, which shows that compared with the method of directly using the random vector, the embedded vector which is more accurately represented can be obtained by extracting the undirected structure information firstly and then extracting the directed structure information.
And then, performing combined analysis of repeated click attention information and the dependency relationship between the items: (a) GCN + GNN, does not consider the different dependencies between the click-repeatedly attention information and the items. (b) AttGCN + GNN, attention information for a repeatedly clicked item is considered only in GCN. (c) GCN + AttGNN, only consider varying degrees of dependency between items in GNN. (d) AttGCN + AttGNN, while fusing repeatedly clicked attention information in GCN and attention-bearing item dependencies in GNN. The experimental results are shown in fig. 6 and 7.
As can be seen from the data in fig. 6 and 7, the AttGCN + AttGNN, which comprehensively considers the repeated click attention information and the inter-item dependency, has the best experimental results on both indicators of the three data sets, which indicates that the repeated click behavior and the inter-item dependency are of certain importance in the recommendation based on the conversation sequence. Meanwhile, according to the experimental expression of the P @20 index in FIG. 6, the attention considering the repeated click attention and the relation between items independently has better expression than the GCN + GNN not considering any attention information, and the AttGCN + AttGNN comprehensively considering the two has the best effect, which shows that the attention information can ensure that important information is kept as much as possible and more accurately expressed as an embedded vector. However, according to the experimental performance of the MRR @20 index in fig. 7, although the AttGCN + AttGNN considering two attention points together still can obtain the best experimental performance, the performance considering one of the two attention points alone is slightly worse than the performance considering GCN + GNN not considering any attention information, that is, the accuracy of the recommendation result can be improved by using the attention information alone, but the prediction effect on the recommendation ranking is not good. The possible reason is that if one model considers attention and the other model does not consider when the vectors are input into the GCN and GNN models, the expression patterns of the adjacent matrixes in the two models are not unified, so that the vectors cannot simultaneously use attention to retain structural information when being input and output between the two models, and the structural information is interfered because of the inconsistent attention patterns, so that an embedded vector with accurate representation is not generated finally, and an accurate prediction score of each item cannot be calculated in the prediction stage.
In the recommendation scene based on the conversation sequence, the repeated click behavior of the user and the graph structure information are both considerable and considerable, because the behavior of the user can be well predicted without knowing the historical preference of the user. The patent refers to the field of 'electric digital data processing'. The invention not only utilizes GCN and GNN models to extract the directed structure information and the undirected structure information in the conversation sequence diagram and carry out linear combination, but also introduces an attention mechanism when generating the hidden vector of the project, effectively extracts the repeated click of the user and the complex conversion information between the projects, and leads the generated conversation vector to be predicted more accurately in the recommendation process. On the actual data sets in three realistic scenarios, the present invention verifies that the proposed algorithm is superior to other most advanced methods, and verifies the effectiveness of the attention mechanism and the complex structural information through exhaustive ablation experiments.
Finally, it should be noted that while the above describes a preferred embodiment of the invention, it will be appreciated by those skilled in the art that, once the basic inventive concepts have been learned, numerous changes and modifications may be made without departing from the principles of the invention, which shall be deemed to be within the scope of the invention. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Claims (10)

1.一种基于有向和无向结构信息的图神经网络的项目推荐方法，其特征在于，所述方法包括：1. A method for project recommendation based on a graph neural network of directed and undirected structural information, wherein the method comprises: S1、令V代表所有会话序列中出现过的项目的集合，即
Figure FDA0003385139960000011
那么一条长度为n的匿名会话序列就可以用
Figure FDA0003385139960000012
表示，且会话s中的项目是按时间先后顺序排列的，每个vi∈V代表了用户在会话s中点击的项目；S1. Let V represent the set of items that have appeared in all session sequences, namely
Figure FDA0003385139960000011
Then an anonymous session sequence of length n can be used
Figure FDA0003385139960000012
represents, and the items in the session s are arranged in chronological order, and each v i ∈ V represents the item clicked by the user in the session s; S2、接收历史会话序列，将所述历史会话序列转换为有向会话序列图Gs＝(Vs,εs,As)，其中Vs代表点集，εs代表边集，As代表邻接矩阵的集合，将As定义为三个邻接矩阵
Figure FDA0003385139960000013
Figure FDA0003385139960000014
和
Figure FDA0003385139960000015
的拼接，其中，
Figure FDA0003385139960000016
代表的是无向图的带权重邻接矩阵，而
Figure FDA0003385139960000017
和
Figure FDA0003385139960000018
分别代表的是带权重的入度邻接矩阵和出度邻接矩阵；S2. Receive a historical session sequence, and convert the historical session sequence into a directed session sequence graph G s =(V s ,ε s ,A s ), where V s represents a point set, ε s represents an edge set, and A s represents a The set of adjacency matrices, defining A s as three adjacency matrices
Figure FDA0003385139960000013
Figure FDA0003385139960000014
and
Figure FDA0003385139960000015
splicing, where,
Figure FDA0003385139960000016
represents the weighted adjacency matrix of the undirected graph, and
Figure FDA0003385139960000017
and
Figure FDA0003385139960000018
respectively represent the weighted in-degree adjacency matrix and out-degree adjacency matrix; S3、把结点vi∈V映射到随机嵌入向量空间中得到d维向量表示
Figure FDA0003385139960000019
利用图卷积网络提取会话序列图中项目的第一中间隐含向量，利用门控图神经网络提取会话序列图中项目转换的第二中间隐含向量，通过第一线性变换得到项目隐含向量；S3. Map the node v i ∈ V to the random embedding vector space to obtain a d-dimensional vector representation
Figure FDA0003385139960000019
Use the graph convolutional network to extract the first intermediate latent vector of the item in the session sequence diagram, use the gated graph neural network to extract the second intermediate latent vector of the item transformation in the session sequence diagram, and obtain the item latent vector through the first linear transformation ; S4、将项目隐含向量输入目标注意力网络，从而得到会话序列s基于目标注意力的向量；S4. Input the item latent vector into the target attention network, so as to obtain the target attention-based vector of the session sequence s; S5、获取会话序列s的全局信息和局部信息，通过第二线性变换构造出会话向量表示；S5. Obtain the global information and local information of the session sequence s, and construct a session vector representation through the second linear transformation; S6、通过softmax函数对会话序列s预测所有目标项目被点击的概率，从而推荐概率大的项目。S6. Predict the probability of all target items being clicked on the session sequence s through the softmax function, thereby recommending items with high probability. 2.根据权利1所述的基于有向和无向结构信息的图神经网络的项目推荐方法，其特征在于，利用图卷积网络提取会话序列图中项目的第一中间隐含向量的步骤如下：2. the item recommendation method based on the graph neural network of directed and undirected structural information according to claim 1, is characterized in that, the step that utilizes graph convolutional network to extract the first intermediate latent vector of item in conversation sequence graph is as follows : S31、生成会话序列图的特征矩阵X；会话序列图中的每个结点vi对应的d维特征向量
Figure FDA0003385139960000021
的堆叠构成会话序列图的特征矩阵
Figure FDA0003385139960000022
Figure FDA0003385139960000023
Figure FDA0003385139960000024
S31. Generate a feature matrix X of the session sequence diagram; a d-dimensional feature vector corresponding to each node v i in the session sequence diagram
Figure FDA0003385139960000021
The stacking of the feature matrices that form the session sequence graph
Figure FDA0003385139960000022
Figure FDA0003385139960000023
Figure FDA0003385139960000024
 S32、对于第k层的图卷积层，用矩阵H(k-1)表示所有结点的输入向量，用H(k)表示结点的输出向量，其中，最初的d维结点向量就是初始输入到图卷积层网络首层的特征，公式为：S32. For the graph convolution layer of the kth layer, use the matrix H (k-1) to represent the input vector of all nodes, and use H (k) to represent the output vector of the node, where the initial d-dimensional node vector is The initial input to the features of the first layer of the graph convolutional layer network, the formula is: H(0)＝X, (1)H (0) = X, (1) 每个图卷积层的输入前，每个结点vi的特征都跟它的局部邻居的特征向量进行均值化，计算公式为：Before the input of each graph convolution layer, the feature of each node v i is averaged with the feature vector of its local neighbors, and the calculation formula is:
Figure FDA0003385139960000025
Figure FDA0003385139960000025
 其中，aij为结点vi和vj之间的边权重，di＝∑jaij；Among them, a ij is the edge weight between nodes v i and v j , d i =∑ j a ij ; S33、图卷积网络的输出为第一中间隐含向量。S33. The output of the graph convolution network is the first intermediate implicit vector. 3.根据权利要求2所述的项目推荐方法，其特征在于，利用图卷积网络提取会话序列图中项目的第一中间隐含向量的步骤如下：3. The item recommendation method according to claim 2, wherein the step of using a graph convolutional network to extract the first intermediate implicit vector of the item in the session sequence diagram is as follows:
Figure FDA0003385139960000026
Figure FDA0003385139960000026
Figure FDA0003385139960000027
Figure FDA0003385139960000027
Figure FDA0003385139960000028
Figure FDA0003385139960000028
 其中S代表“对称归一化”后带自环的邻接矩阵，
Figure FDA0003385139960000029
并且
Figure FDA0003385139960000031
是
Figure FDA0003385139960000032
的度矩阵，⊙是点乘运算符，
Figure FDA0003385139960000033
为图卷积网络的传播矩阵，α和β为超参数，I是单位矩阵，
Figure FDA0003385139960000034
为输出的第一中间隐含向量。where S represents the adjacency matrix with self-loop after "symmetric normalization",
Figure FDA0003385139960000029
and
Figure FDA0003385139960000031
Yes
Figure FDA0003385139960000032
The degree matrix of , ⊙ is the dot product operator,
Figure FDA0003385139960000033
is the propagation matrix of the graph convolutional network, α and β are hyperparameters, I is the identity matrix,
Figure FDA0003385139960000034
is the first intermediate implicit vector of the output. 4.根据权利要求3所述的项目推荐方法，其特征在于，对于有边相连的项目，通过提高公式(4)的左半部分的权重，提高矩阵中自我信息的注意力，超参α和超参β用于控制传播矩阵信息和单位阵信息的比例，从而控制传播过程中带注意力的结点信息的吸收比例。4. The item recommendation method according to claim 3, characterized in that, for items connected by edges, by increasing the weight of the left half of formula (4), the attention of self-information in the matrix is improved, and the hyperparameter α and The hyperparameter β is used to control the ratio of the propagation matrix information to the identity matrix information, thereby controlling the absorption ratio of the node information with attention in the propagation process. 5.根据权利2-4任一权利要求所述的项目推荐方法，其特征在于，利用门控图神经网络提取会话序列图中项目转换的第二中间隐含向量的步骤为：5. The item recommendation method according to any one of claims 2-4, wherein the step of extracting the second intermediate latent vector of item conversion in the session sequence diagram by using a gated graph neural network is:
Figure FDA0003385139960000035
Figure FDA0003385139960000035
Figure FDA0003385139960000036
Figure FDA0003385139960000036
Figure FDA0003385139960000037
Figure FDA0003385139960000037
Figure FDA0003385139960000038
Figure FDA0003385139960000038
Figure FDA0003385139960000039
Figure FDA0003385139960000039
 其中，
Figure FDA00033851399600000310
和
Figure FDA00033851399600000311
用于控制权重和偏置项的大小，
Figure FDA00033851399600000312
用来表示结点与相邻结点间的相互作用的结果，
Figure FDA00033851399600000313
和
Figure FDA00033851399600000314
分别是重置门和更新门，权重矩阵Wz、Uz，Wr、Ur和Wo、Uo分别代表了重置门、更新门和输出门中可学习的网络参数，
Figure FDA00033851399600000315
代表了结点vi的第二中间隐含向量，
Figure FDA00033851399600000316
是会话中的结点向量序列，且
Figure FDA00033851399600000317
Figure FDA00033851399600000318
为图卷积网络输出的第一中间隐含向量，σ(·)是sigmoid函数，⊙是点乘运算符；邻接矩阵
Figure FDA00033851399600000319
Figure FDA00033851399600000320
代表图中结点的交流，
Figure FDA00033851399600000321
代表结点vi在
Figure FDA00033851399600000322
中的两列矩阵块；矩阵
Figure FDA0003385139960000041
定义为入度矩阵
Figure FDA0003385139960000042
和出度矩阵
Figure FDA0003385139960000043
的拼接。in,
Figure FDA00033851399600000310
and
Figure FDA00033851399600000311
is used to control the size of the weights and bias terms,
Figure FDA00033851399600000312
is used to represent the result of the interaction between a node and adjacent nodes,
Figure FDA00033851399600000313
and
Figure FDA00033851399600000314
are the reset gate and the update gate, respectively. The weight matrices W z , U z , W r , Ur and W o , U o represent the learnable network parameters in the reset gate, update gate and output gate, respectively,
Figure FDA00033851399600000315
represents the second intermediate implicit vector of node v i ,
Figure FDA00033851399600000316
is the sequence of node vectors in the session, and
Figure FDA00033851399600000317
Figure FDA00033851399600000318
is the first intermediate latent vector output by the graph convolutional network, σ( ) is the sigmoid function, ⊙ is the dot product operator; the adjacency matrix
Figure FDA00033851399600000319
Figure FDA00033851399600000320
represents the communication of nodes in the graph,
Figure FDA00033851399600000321
represents the node v i in
Figure FDA00033851399600000322
A two-column matrix block in ; matrix
Figure FDA0003385139960000041
Defined as an in-degree matrix
Figure FDA0003385139960000042
and out-degree matrix
Figure FDA0003385139960000043
splicing. 6.根据权利要求1所述的项目推荐方法，其特征在于，所述步骤S3中，第一线性变换为：6. The item recommendation method according to claim 1, wherein in the step S3, the first linear transformation is:
Figure FDA0003385139960000044
Figure FDA0003385139960000044
 其中γ为超参数，用于平衡带注意力的无向结构信息与有向结构信息的比例；H为项目隐含向量，
Figure FDA0003385139960000045
为门控图神经网络的输出。where γ is a hyperparameter used to balance the ratio of undirected structural information with attention to directed structural information; H is the item latent vector,
Figure FDA0003385139960000045
is the output of the gated graph neural network. 7.根据权利要求1所述的项目推荐方法，其特征在于，所述步骤S4中，得到会话序列s基于目标注意力的向量的步骤如下：7. The item recommendation method according to claim 1, wherein in the step S4, the step of obtaining the vector of the conversation sequence s based on the target attention is as follows: S41、利用局部目标注意力模型计算会话序列s中所有项目vi对每个目标项目vt∈V的注意力分布ei,t，再通过softmax(·)函数获得注意力分数βi,t，公式为：S41. Use the local target attention model to calculate the attention distribution e i ,t of all items vi in the session sequence s to each target item v t ∈ V, and then obtain the attention score β i,t through the softmax(·) function , the formula is:
Figure FDA0003385139960000046
Figure FDA0003385139960000046
 其中，
Figure FDA0003385139960000047
和
Figure FDA00033851399600000414
分别为所有项目vi与目标项目vt的项目隐含向量表示，
Figure FDA0003385139960000048
为带权重矩阵，通过训练生成，exp(·)为计算常数e的n次幂函数；in,
Figure FDA0003385139960000047
and
Figure FDA00033851399600000414
are the item latent vector representations of all items vi and target item v t , respectively,
Figure FDA0003385139960000048
is a weighted matrix, generated through training, and exp( ) is the nth power function for calculating the constant e; S42、对于每个会话序列s，用户对目标项目vt的兴趣可以表示为
Figure FDA0003385139960000049
计算公式为：S42. For each session sequence s, the user's interest in the target item v t can be expressed as
Figure FDA0003385139960000049
The calculation formula is:
Figure FDA00033851399600000410
Figure FDA00033851399600000410
 8.根据权利要求7所述的项目推荐方法，其特征在于，所述步骤S5包括：8. The item recommendation method according to claim 7, wherein the step S5 comprises: S51、获取局部向量
Figure FDA00033851399600000411
所述局部向量为会话序列s中最后一个项目
Figure FDA00033851399600000412
的向量表示，
Figure FDA00033851399600000413
即为其隐含向量，公式为：S51. Obtain a local vector
Figure FDA00033851399600000411
The local vector is the last item in the session sequence s
Figure FDA00033851399600000412
the vector representation of ,
Figure FDA00033851399600000413
That is, its implicit vector, the formula is:
Figure FDA0003385139960000051
Figure FDA0003385139960000051
 S52、获取全局向量
Figure FDA0003385139960000052
所述全局向量聚合了会话序列s中所有出现的项目向量，还利用注意力机制引入与整个会话序列s中出现的项目[v1,v2,…,vn]之间的依赖关系，公式为：S52, obtain the global vector
Figure FDA0003385139960000052
The global vector aggregates all the item vectors that appear in the session sequence s, and also uses the attention mechanism to introduce dependencies with the items [v 1 , v 2 ,..., v n ] that appear in the entire session sequence s, the formula for:
Figure FDA0003385139960000053
Figure FDA0003385139960000053
Figure FDA0003385139960000054
Figure FDA0003385139960000054
 其中
Figure FDA0003385139960000055
为会话中最后一个项目，
Figure FDA0003385139960000056
且
Figure FDA0003385139960000057
是相应的权重参数，αi代表最后一项与整个会话序列中出现的项目之间的依赖关系；in
Figure FDA0003385139960000055
is the last item in the session,
Figure FDA0003385139960000056
and
Figure FDA0003385139960000057
is the corresponding weight parameter, α i represents the dependency between the last item and the items appearing in the entire session sequence; S53、将局部向量、全局向量和基于目标注意力的向量拼接，并利用第二线性转换得到会话序列s所对应的会话向量，公式为：S53, splicing the local vector, the global vector and the vector based on the target attention, and using the second linear transformation to obtain the session vector corresponding to the session sequence s, the formula is:
Figure FDA0003385139960000058
Figure FDA0003385139960000058
 其中权重参数
Figure FDA0003385139960000059
把三个向量拼接的结果投射到向量空间
Figure FDA00033851399600000510
中；where the weight parameter
Figure FDA0003385139960000059
Project the result of concatenating three vectors into a vector space
Figure FDA00033851399600000510
middle; 所述W1、W2、W3是权重参数矩阵，通过训练生成。The W1, W2, and W3 are weight parameter matrices, which are generated through training. 9.根据权利要求8所述的项目推荐方法，其特征在于，所述步骤S6包括：9. The item recommendation method according to claim 8, wherein the step S6 comprises: S61、在得到了每个会话序列s对应的会话向量sh之后，对于所有的项目vi∈V的得分
Figure FDA00033851399600000511
进行计算，即将所有项目vi∈V的第二隐含向量
Figure FDA00033851399600000512
与其相对应的会话向量sh进行相乘，公式如下：S61. After obtaining the session vector sh corresponding to each session sequence s, the scores for all items v i ∈ V
Figure FDA00033851399600000511
Do the computation, i.e. the second latent vector of all items v i ∈ V
Figure FDA00033851399600000512
Multiply it with the corresponding session vector sh, the formula is as follows:
Figure FDA00033851399600000513
Figure FDA00033851399600000513
 S62、通过第二softmax函数输出向量
Figure FDA00033851399600000514
公式如下：S62, output the vector through the second softmax function
Figure FDA00033851399600000514
The formula is as follows:
Figure FDA0003385139960000061
Figure FDA0003385139960000061
 其中
Figure FDA0003385139960000062
代表了所有目标项目的预测推荐得分，而
Figure FDA0003385139960000063
代表了目标项目在会话序列s下一时刻被点击的概率,
Figure FDA0003385139960000064
中排名最高的前K个的项目即为将要推荐的项目；其中，对于会话序列图Gs，损失函数定义成预测值与实际值的交叉熵。in
Figure FDA0003385139960000062
represents the predicted recommendation scores of all target items, and
Figure FDA0003385139960000063
represents the probability that the target item is clicked at the next moment of the session sequence s,
Figure FDA0003385139960000064
The top K items with the highest ranking are the items to be recommended; among them, for the session sequence graph G s , the loss function is defined as the cross entropy between the predicted value and the actual value. 10.根据权利要求1-9所述的项目推荐方法，其特征在于，重复执行步骤S2-S6，使用基于时间的反向传播算法训练参数。10. The item recommendation method according to claims 1-9, wherein steps S2-S6 are repeatedly performed, and parameters are trained using a time-based back-propagation algorithm.
CN202111447363.6A 2021-12-01 2021-12-01 An item recommendation method based on graph neural network with directed and undirected structural information Active CN114117229B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111447363.6A CN114117229B (en) 2021-12-01 2021-12-01 An item recommendation method based on graph neural network with directed and undirected structural information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111447363.6A CN114117229B (en) 2021-12-01 2021-12-01 An item recommendation method based on graph neural network with directed and undirected structural information

Publications (2)

Publication Number Publication Date
CN114117229A true CN114117229A (en) 2022-03-01
CN114117229B CN114117229B (en) 2025-05-27

Family

ID=80369156

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111447363.6A Active CN114117229B (en) 2021-12-01 2021-12-01 An item recommendation method based on graph neural network with directed and undirected structural information

Country Status (1)

Country Link
CN (1) CN114117229B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114897145A (en) * 2022-05-20 2022-08-12 支付宝(杭州)信息技术有限公司 Sequence recommendation method, system, device and medium
CN116952554A (en) * 2023-07-05 2023-10-27 北京科技大学 Multi-sensor mechanical equipment fault diagnosis method and device based on graph rolling network
CN118210966A (en) * 2024-02-26 2024-06-18 北京市遥感信息研究所 A location-enhanced and time-aware human online behavior prediction method and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816101A (en) * 2019-01-31 2019-05-28 中科人工智能创新技术研究院(青岛)有限公司 A kind of session sequence of recommendation method and system based on figure convolutional neural networks
CN110119467A (en) * 2019-05-14 2019-08-13 苏州大学 A kind of dialogue-based item recommendation method, device, equipment and storage medium
CN112035746A (en) * 2020-09-01 2020-12-04 湖南大学 A Conversation Recommendation Method Based on Spatiotemporal Sequence Graph Convolutional Networks
CN112364976A (en) * 2020-10-14 2021-02-12 南开大学 User preference prediction method based on session recommendation system
CN113326425A (en) * 2021-04-20 2021-08-31 中国电子科技集团公司第五十四研究所 Session recommendation method and system based on structure and semantic attention stacking

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816101A (en) * 2019-01-31 2019-05-28 中科人工智能创新技术研究院(青岛)有限公司 A kind of session sequence of recommendation method and system based on figure convolutional neural networks
CN110119467A (en) * 2019-05-14 2019-08-13 苏州大学 A kind of dialogue-based item recommendation method, device, equipment and storage medium
CN112035746A (en) * 2020-09-01 2020-12-04 湖南大学 A Conversation Recommendation Method Based on Spatiotemporal Sequence Graph Convolutional Networks
CN112364976A (en) * 2020-10-14 2021-02-12 南开大学 User preference prediction method based on session recommendation system
CN113326425A (en) * 2021-04-20 2021-08-31 中国电子科技集团公司第五十四研究所 Session recommendation method and system based on structure and semantic attention stacking

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
孙鑫;刘学军;李斌;梁珂;: "基于图神经网络和时间注意力的会话序列推荐", 计算机工程与设计, no. 10, 16 October 2020 (2020-10-16), pages 220 - 227 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114897145A (en) * 2022-05-20 2022-08-12 支付宝(杭州)信息技术有限公司 Sequence recommendation method, system, device and medium
CN116952554A (en) * 2023-07-05 2023-10-27 北京科技大学 Multi-sensor mechanical equipment fault diagnosis method and device based on graph rolling network
CN116952554B (en) * 2023-07-05 2024-11-12 北京科技大学 Multi-sensor mechanical equipment fault diagnosis method and device based on graph convolutional network
CN118210966A (en) * 2024-02-26 2024-06-18 北京市遥感信息研究所 A location-enhanced and time-aware human online behavior prediction method and system

Also Published As

Publication number Publication date
CN114117229B (en) 2025-05-27

Similar Documents

Publication Publication Date Title
Wu et al. Session-based recommendation with graph neural networks
CN111523047B (en) Multi-relation collaborative filtering algorithm based on graph neural network
CN112150210B (en) Improved neural network recommendation method and system based on GGNN (global warming network)
CN110119467B (en) Project recommendation method, device, equipment and storage medium based on session
CN111797321B (en) A method and system for personalized knowledge recommendation for different scenarios
CN112364976B (en) User preference prediction method based on session recommendation system
Gwadabe et al. Improving graph neural network for session-based recommendation system via non-sequential interactions
CN112733018B (en) A conversation recommendation method based on graph neural network GNN and multi-task learning
CN114117229B (en) An item recommendation method based on graph neural network with directed and undirected structural information
CN111581520A (en) Item Recommendation Method and System Based on Item Importance in Session
CN110796313B (en) Session recommendation method based on weighted graph volume and item attraction model
CN111080400A (en) A product recommendation method, system and storage medium based on gated graph convolutional network
CN111753209A (en) A method for generating sequence recommendation list based on improved temporal convolutional network
CN113051468B (en) Movie recommendation method and system based on knowledge graph and reinforcement learning
CN115470406A (en) Graph neural network session recommendation method based on dual-channel information fusion
CN112765461A (en) Session recommendation method based on multi-interest capsule network
Chen et al. Combine temporal information in session-based recommendation with graph neural networks
CN113010774A (en) Click rate prediction method based on dynamic deep attention model
CN114625969A (en) Recommendation method based on interactive neighbor session
CN117056609A (en) Session recommendation method based on multi-layer aggregation enhanced contrast learning
CN115510322A (en) Multi-objective optimization recommendation method based on deep learning
CN114547276A (en) Three-channel diagram neural network-based session recommendation method
CN114925270A (en) Session recommendation method and model
CN116051175A (en) Click-through rate prediction model and prediction method based on deep multi-interest network
Wang et al. Session-based recommendation with time-aware neural attention network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant