CN112990972B - Recommendation method based on heterogeneous graph neural network - Google Patents

Recommendation method based on heterogeneous graph neural network Download PDF

Info

Publication number
CN112990972B
CN112990972B CN202110296182.1A CN202110296182A CN112990972B CN 112990972 B CN112990972 B CN 112990972B CN 202110296182 A CN202110296182 A CN 202110296182A CN 112990972 B CN112990972 B CN 112990972B
Authority
CN
China
Prior art keywords
commodity
node
user
graph
heterogeneous
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110296182.1A
Other languages
Chinese (zh)
Other versions
CN112990972A (en
Inventor
许勇
邵逸臻
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202110296182.1A priority Critical patent/CN112990972B/en
Publication of CN112990972A publication Critical patent/CN112990972A/en
Application granted granted Critical
Publication of CN112990972B publication Critical patent/CN112990972B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements
    • G06Q30/0255—Targeted advertisements based on user history
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9536—Search customisation based on social or collaborative filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0241—Advertisements
    • G06Q30/0251—Targeted advertisements

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Development Economics (AREA)
  • General Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Data Mining & Analysis (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Game Theory and Decision Science (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention belongs to the technical field of recommendation systems, and relates to a recommendation method based on a heterogeneous graph neural network, which comprises the following steps: collecting a data set with the social relationship among users, the historical data of user-commodity interaction and commodity category information, filtering invalid data and carrying out negative sampling; randomly selecting a user set and a related commodity set, and carrying out multi-level graph sampling and graph building; and (3) node feature extraction: inputting the constructed graph into a heterogeneous graph neural network for processing to obtain a fusion node embedding vector of the nodes; for the commodity nodes which do not need to be subjected to the recalibration step, the fusion node embedded vectors of the commodity nodes are commodity fusion embedded vectors; recalibration: recalibrating the user fusion node embedded vector to obtain a user final representation embedded vector; and performing preference prediction by using the user final expression embedded vector and the commodity fusion embedded vector, and obtaining a recommendation sequence. The method solves the problems of data sparseness and data loss, and has the advantages of accurate recommendation and the like.

Description

Recommendation method based on heterogeneous graph neural network
Technical Field
The invention belongs to the technical field of recommendation systems, and relates to a recommendation method based on a heterogeneous graph neural network.
Background
In the current information explosion era, information received by users every day exceeds the range which can be processed by individuals, and the selection of the needed relevant useful information by the users is seriously disturbed by massive irrelevant redundant information. The recommendation system is an application for recommending commodities to a target user according to user historical behaviors and personal preferences, and can provide personalized more useful related information for the user, so that the problem of information overload is effectively relieved.
In recent years, with the explosive growth of the internet and smart mobile devices, recommendation systems have been widely applied to various network services such as e-commerce websites, online forums, advertisement platforms, and video websites. The quality of the recommendation method directly determines whether the personalized recommendation of the recommendation system is accurate or not. However, the conventional recommendation method based on the collaborative filtering model has the problems of data sparseness and the like, so that some long-tail commodities are difficult to recommend to users in need, and the requirement of the users for rapidly and accurately acquiring the needed commodities is greatly limited. Therefore, in order to alleviate the problem caused by data sparseness, other information besides the interaction between the user and the commodity is introduced, and it is increasingly critical to research the recommendation method under the heterogeneous relationship.
Current recommendation methods can be simply divided into three categories: collaborative filtering methods, content-based methods, and hybrid methods. The collaborative filtering method not only utilizes the historical behaviors of the target user to model the user, but also utilizes other users and commodities to predict the target user-commodity interaction. User-commodity interaction history data can generally provide better prediction results and is easy to acquire, and therefore is generally regarded as core information of a recommendation method. Although the collaborative filtering method has a good prediction performance, the problem of data sparseness inevitably exists in the interaction historical data of the user and the commodity, so that the collaborative filtering method cannot fundamentally deal with the recommendation and cold start problems of the long-tail commodity. The content-based approach models using data describing the user or the good, such as the user's age, gender, or category of the good, textual description, etc., such data primarily describing features of the user or the good itself, rather than focusing on the user's interaction with the good, and is therefore referred to as content data. The data can not be scarce due to new users or the use frequency of the users and the like, and the data sparseness problem can be effectively avoided. The mixing method is to fuse the collaborative filtering method and the content-based method, not only use the user-commodity interaction historical data more fitting the recommendation scene, but also use the characteristics of the abundant users and commodities, and generally obtain more accurate recommendation effect.
With the rapid development of deep learning technology, deep learning methods have been widely applied to various new recommendation methods. As the graph Neural network collaborative filtering model NGCF (described in Wang, xiang, et al, "Neural graph collaborative filtering." Proceedings of the 42nd interfacial ACM SIGUR coherence Research and depth in Information retrieval.2019 ") is widely used and the LightGCN (described in He, xiangnan, et al," Lightgcn: simple and power mapping coherence calculation for recording network, "Proceedings of the 43rd International ACM SIGUR coherence Research and Development in Information retrieval.) described in Wang et al has been applied to the methods of video network service, the performance of the graph Neural network collaborative filtering model NGCF has been increasingly studied. Although the graph neural network can achieve excellent achievement on the collaborative filtering method, the graph neural network still suffers from the data sparseness problem of the collaborative filtering method. Therefore, in order to reduce the problems caused by data sparseness, it is necessary to merge more content information and user-related social information into the graph model.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a recommendation method based on a heterogeneous graph neural network.
The invention is realized by adopting the following technical scheme:
a recommendation method based on a heterogeneous graph neural network comprises the following steps:
collecting a data set with the social relationship among users, user-commodity interaction historical data and commodity category information, filtering invalid data and carrying out negative sampling;
randomly selecting a user set and a related commodity set from the data set, and carrying out multi-level graph sampling and graph building;
node feature extraction: inputting the constructed graph into a heterogeneous graph neural network comprising a plurality of heterogeneous graph memory network layers for processing to obtain a fusion node embedding vector of nodes; for the commodity nodes which do not need to be subjected to the recalibration step, the fusion node embedded vectors of the commodity nodes are the commodity fusion embedded vectors;
recalibration: recalibrating the user fusion node embedded vector to obtain a user final representation embedded vector;
and (3) recommendation and prediction: and performing preference prediction by using the user final representation embedding vector and the commodity fusion embedding vector, and obtaining a recommendation sequence.
Preferably, the multi-level graph sampling and mapping process includes:
(1) Randomly selecting a user commodity pair of the data set, if the commodity pair is not subjected to negative sampling, further performing commodity negative sampling, and taking the user, the commodity and the negative sampling commodity in the selected set as seed nodes of graph sampling;
(2) And carrying out multi-order graph sampling on the seed nodes, determining the order and the number of sampling points of each order, and then carrying out graph sampling.
Preferably, the heterogeneous neural network comprises a multi-layer hopping junction neural network and a user-embedded vector recalibration network, wherein:
the multi-layer jump connection neural network uses a multi-layer jump structure, and each layer of the network layer of the layer graph has a jump connection to the final output;
and the user embedded vector recalibration network is used for recalibrating the user fusion node embedded vectors.
Preferably, the step of extracting the fusion node embedding vector of the node comprises:
(1) Inputting the graph constructed in the step S2 into a neural network of the heterogeneous graph, and obtaining an initial embedded vector of the node through an initial embedded vector table;
(2) Obtaining a node output embedded vector of each layer of heterogeneous graph memory network layer of the nodes through a plurality of layers of heterogeneous graph memory network layers;
(3) And fusing the initial embedded vectors of the nodes and the node output embedded vectors of each layer of the heteromorphic image memory network layer to obtain fused node embedded vectors of the nodes.
Preferably, the processing procedure of the heterogeneous graph memory network layer comprises the following steps:
1) Extracting the characteristics of an edge end point of each edge in the graph by using a memory enhanced heterogeneous relation encoder;
2) Fusing the messages received by each node to obtain node output embedded vectors;
3) Directly taking the output embedded vector after fusing each node message as a node output embedded vector output by a heterogeneous graph memory network layer; or, using Layernorm for standardization and adding the output of the memory enhanced heterogeneous relation encoder using the self-loop as the node output embedded vector of the final output of the heterogeneous graph memory network layer.
Preferably, the memory-enhanced heterogeneous relationship encoder comprises the operations of:
a) Obtaining the coefficient of each memory unit of the memory enhanced heterogeneous relation encoder of the lower end point of the corresponding relation of the edges by using the bias linear transformation of the learnable band after the activation function is activated;
b) And performing linear transformation on the embedded vector of the lower starting point of the corresponding relation of the sides by using the sum of the products of all the memory units and the corresponding coefficients thereof as a transformation matrix.
Preferably, the representation form of the message fusion function for fusing the messages received by each node is different among different types of nodes, and the representation form includes:
for the user node, the message fusion function is expressed as the sum of the normalized social relationship of the user node and the messages obtained by the user-commodity interaction history through the memory enhanced heterogeneous relationship encoder;
for the commodity node, the message fusion function of the commodity node is expressed as the sum of the normalized user-commodity interaction history of the commodity node and the message obtained by the commodity generic relation respectively through the memory enhanced heterogeneous relation encoder;
for the commodity category node, the message fusion function of the commodity category node is expressed as the sum of messages obtained by a memory enhanced heterogeneous relation encoder of the normalized commodity category node commodity category relation.
Preferably, the method for obtaining the fusion node embedding vector of the node comprises the following steps: and splicing the node initial embedded vector with the output of each layer of the heteromorphic graph memory network layer, and normalizing by using Layernorm.
Preferably, the recalibration method comprises: and adding the user fusion node embedded vector and the result of the self-looping first-order graph convolution of the user node on the social relationship to obtain a user final representation embedded vector.
Preferably, the recommendation method further comprises:
the heterogeneous graph neural network training phase uses BPRLoss as a loss function to supervise and then backpropagate the gradient into the heterogeneous graph neural network.
Compared with the prior art, the invention has the following advantages:
1. according to the method and the device, the social relationship among users, the characteristics of commodities and the interaction historical data of the users and the commodities are considered at the same time, so that the recommendation method is ensured to achieve the excellent performance of the collaborative filtering recommendation method, the advantage of avoiding data sparseness based on the content recommendation method is achieved, the influence of the social relationship on the preference of the users is considered, more accurate recommendation is achieved, and the problem of data overload is solved.
2. The method can well solve the problem of data sparsity encountered by the collaborative filtering method. For users and commodities with rare interaction historical data, the traditional collaborative filtering method is limited to the neighbors of the target object in the depiction of the objects, but the rare number of the neighbors can bring deviation to the depiction of the objects. The social information and the category information are introduced, so that the characteristics of the target object can be described from the perspective of the user and the commodity, and the problems of data sparseness and data loss in the collaborative filtering method are solved.
3. The invention introduces a jump connection mode to overcome the gradient disappearance problem in the traditional graph neural network architecture, and each graph neural network layer is provided with a jump connection to the final output layer, so that the gradient disappearance problem can be avoided when the network deepens.
4. The heterogeneous graph neural network layer of the invention adopts a memory-enhanced relationship encoder to capture heterogeneous relationship features. Different memory parameters are adopted among different types of connections, and parameter level differentiation is provided for message transmission of the neural network of the heterogeneous graph, so that mutual information among different types of messages is less, and the information quantity of message transmission is increased, and therefore, the heterogeneous relationships of user social information, commodity category information and the like can be better utilized to recommend commodities to users.
5. The invention adopts a user embedded vector recalibration mechanism, the characteristic component sources of the user node embedded vectors output by the graph neural network are complex, and the recalibration can enable the user fusion node embedded vectors to be greatly influenced by social connection, so that the user finally shows that the embedded vectors have stronger character feature description capacity.
Drawings
FIG. 1 is a flow diagram of a recommendation method in one embodiment;
FIG. 2 is a diagram illustrating node feature extraction according to an embodiment;
FIG. 3 is a diagram of a heterogeneous patterned neural network architecture in one embodiment.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments.
The working principle of the invention comprises: firstly, preprocessing a data set to obtain social relations among users, user-commodity interaction historical data and commodity category data, sampling a multi-level graph and constructing a heterogeneous graph, and inputting the constructed heterogeneous graph into a trained heterogeneous graph network for recommendation and prediction.
A recommendation method based on a heterogeneous graph neural network comprises the following steps:
s1, constructing a data set, collecting the data set with the social relationship among users, the historical data of user-commodity interaction and commodity category information in the scenes of e-commerce, comment websites and the like, filtering invalid data and carrying out negative sampling.
Most of the currently disclosed recommendation methods are based on collaborative filtering methods, but in real e-commerce, comment and video network services, a service provider usually has other information besides user-commodity interaction history information such as social relations among users and commodity categories. Therefore, if the additional information can be utilized, the characteristics of the user and the commodity can be accurately described, so that more accurate user preference can be obtained, the time spent by the user on the commodity which is not interested is reduced, and the interaction willingness of the user is improved. While many neural network methods based on collaborative filtering perform graph neural network reasoning according to a user-commodity interaction bipartite graph, the homograph method based on a single relationship is not applicable to the invention containing multiple relationships. Therefore, if such additional information is to be utilized, there is a need for the collection of data sets that, in addition to the historical data of user-item interactions, also contain user social relationships and item category data, as used in common collaborative filtering methods. The collection of data sets is thus critical to network training.
When the data set is constructed, the data set with the social relationship among users, the user-commodity interaction historical data and the commodity category information in the scenes of a merchant website, a comment website and the like can be collected. Several review data sets, ciao, eponions, yelp, that have been published so far, can be used as candidate data sets for model training and validation tests. In addition, a data set having more relationships, such as a data set containing personal characteristics of a user or a data set containing interrelationships between commodities, may also be used as a model target data set. According to work that is currently available, the collection of data sets may be done as follows:
(1) The existing public data sets, such as Ciao, epionins, yelp data sets and the like, are directly collected, the data sets comprise the relations, and the data sets can be directly used in the neural network of the heterogeneous graph after simple preprocessing.
(2) The data set is collected or generated by the network service provider, the data can be collected by the network service provider, and the user social relationship or commodity category relationship can be generated by other data such as user-commodity interaction data and the like by a general user.
After data collection, if no cold start requirement exists, users and commodities without user-commodity interaction are removed from the data set when the data set is constructed, and effective data can be prevented from being polluted to a certain extent. And then sequencing the user-commodity interaction historical data according to time, wherein the last two interaction records of each user are respectively used as a verification set and a test set, and the rest interaction historical records are used as training sets. For users with less than three user-commodity interactions, the user-commodity interaction can be avoided from being put into a verification set and a test set. For the validation set and the test set, negative sampling is performed when the data set is constructed to ensure reproducibility of subsequent results.
S2, preprocessing, namely randomly selecting a user set and a related commodity set from the data set, sampling and drawing a multi-level graph, and taking the graph as the input of a neural network of the heterogeneous graph.
The input of the heteromorphic graph neural network is a graph containing social relations among users, user-commodity interaction history and commodity category information, so that a sample is required to be sampled and mapped, and the method mainly comprises the following two steps:
(1) And randomly selecting user commodity pairs of the data set, if the commodity pairs are not subjected to negative sampling, further performing commodity negative sampling, and taking the users, the commodities and the negative sampling commodities in the selected set as seed nodes of the graph sampling.
(2) And carrying out multi-order graph sampling on the seed nodes, determining the order and the number of sampling points of each order, and then carrying out graph sampling. And finally, taking the graph constructed by the sampled nodes and the edges with the relations involved in the sampling process as an input graph of the heterogeneous graph neural network.
And S3, extracting node characteristics, inputting the constructed graph into a heterogeneous graph neural network comprising a plurality of layers of heterogeneous graph memory network layers for processing, and obtaining a fusion node embedded vector of the nodes.
And (4) node feature extraction, namely firstly obtaining an initial embedded matrix by all nodes on the graph constructed by preprocessing according to an initial embedded vector table. And then inputting the initial embedded matrix into a heterogeneous graph memory network layer, and obtaining an output node embedded matrix. Repeating the process for several times to obtain node embedded matrixes under different levels, wherein the output of each layer of heterogeneous graph memory network layer corresponds to different node characteristics. The shallow features better describe the nodes themselves and the neighbor features directly connected to the nodes, while the deep features derive the high-level abstract features of the nodes. And the output of the final heterogeneous graph neural network of a certain node is subjected to fusion by standardizing the initial embedded vector of the node and the embedded vector output by each layer of heterogeneous graph memory network layer to obtain a fusion node embedded vector of the node.
In a preferred embodiment, as shown in FIG. 3, the heterogeneous neural network comprises a multi-layer hopping junction neural network and a user-embedded vector recalibration network, which are connected in series.
The multi-layer hopping nodes used by the multi-layer hopping connection neural network are connected, each layer of graph network layer (heterogeneous graph memory network layer) is connected with the final output in a hopping mode, the problem that the gradient of the network disappears can be avoided, and meanwhile, more layers of information can be reserved by the output of the multi-layer graph neural network. The output of each layer of the graph neural network corresponds to the aggregation information of a plurality of orders of neighbors, the low-order features describe more features of the user or the commodity object, the difference between the features of the nodes of the same type is larger, the distinction between the objects is stronger, the high-order features describe more common features of the nodes in the certain order of the neighborhood of the object, the difference between the features of the nodes of the same type is relatively smaller, and certain class of features and features of the user or the commodity object can be better described. And finally, the input features, the low-order features and the high-order features are fused, and the fused features can better depict the features of the user and the commodity and make the features of the user and the commodity more distinctive.
Specifically, the node features are extracted, and as shown in fig. 2, an initial embedding matrix H is obtained by first passing all the nodes on the graph through an initial embedding vector table (0) And then inputting the graph neural network into a graph neural network comprising a plurality of layers for processing, wherein each layer of the graph neural network is correspondingly embedded with the feature representation in the neighborhood of different orders, the features of the lower order describe more features of the user or the commodity object per se, the features of the higher order describe more common features of nodes in the neighborhood of certain orders of the user or the commodity object, and the final output features are the fusion of the multi-order graph neural network output, so that the multi-layer features of different orders can be obtained through the graph neural network output.
Specifically, the step of extracting the fusion node embedding vector of the node includes:
(1) Inputting the graph constructed in the step S2 into a neural network of the heterogeneous graph, and obtaining an initial embedded vector of the node through an initial embedded vector table;
(2) Through a plurality of layers of heterogeneous graph memory network layers, the operation of each layer of heterogeneous graph memory network layer comprises the following steps:
1) And performing feature extraction of the edge end point on each edge in the graph by using a memory enhanced heterogeneous relation encoder. Wherein the memory-enhanced heterogeneous relationship encoder comprises the operations of:
a) Obtaining the coefficient of each memory unit of the memory enhanced heterogeneous relation encoder of the lower end point of the corresponding relation of the edges by using the bias linear transformation of the learnable band after the activation function is activated;
b) And performing linear transformation on the embedded vector of the lower starting point of the corresponding relation by using the sum of the products of all the memory units and the corresponding coefficients thereof as a transformation matrix.
2) And fusing the messages received by each node, wherein the expression form of the message fusion function in different types of nodes is as follows:
for the user node, the message fusion function is expressed as the sum of the normalized social relationship of the user node and the message obtained by the user-commodity interaction history through the memory enhanced heterogeneous relationship encoder.
For the commodity node, the message fusion function is expressed as the sum of the normalized user-commodity interaction history of the commodity node and the normalized commodity generic relation obtained by the memory enhanced heterogeneous relation encoder.
For the commodity category node, the message fusion function is expressed as the sum of the messages of the normalized commodity category node commodity generic relationship obtained by the memory enhanced heterogeneous relationship encoder.
3) The output embedded vector after the fusion of each node message can be directly used as the node output embedded vector output by the heterogeneous graph memory network layer. Or, furthermore, layernorm can be used for standardization, and the output of a memory enhanced heterogeneous relation encoder using a self-loop is added to serve as a node output embedding vector of the final output of the memory network layer of the heterogeneous graph, so that the stability of the model during training can be enhanced.
(3) And after the node output embedded vectors of each layer of the heteromorphic image memory network layer are obtained, fusing the node output embedded vectors to obtain fusion node embedded vectors of the nodes. The fusion method comprises the following steps: and splicing the initial embedded vectors of the nodes with the output of each layer of the heterogeneous graph memory network layer, and standardizing by using Layernorm.
The specific structure of each layer of the neural network is as follows (taking the l-th layer as an example):
(1) Input diagram G = (G =: (b))V, E, a, B), and an input graph node embedding matrix H (l-1) . Where each node V ∈ V and each edge E ∈ E are associated with its mapping function V → A and E → B, which are a set of point types and a set of edge relationships, respectively. The input graph node embedded matrix is the output of the previous layer of the graph neural network, and the first layer is the initial embedded matrix obtained through the initial embedded vector table.
(2) For a certain node t of the graph, the vector H is embedded in the output of the ith layer of the graph neural network (l) [t]Can be described by the following formula:
Figure GDA0003786233400000111
wherein: n (t) represents the (starting) neighbor set of node t, e s,t Representing an edge connecting node s and node t,
Figure GDA0003786233400000112
for the memory enhancement heterogeneous relation encoder, the Aggre (-) is a message fusion function; h (l-1) [t]Embedding a vector for the input of node t, H (l-1) [s]To pass through the edge e s,t The input of the start point s embeds a vector.
The memory-enhanced heterogeneous relation encoder is used for capturing heterogeneous relation characteristics, and different memory units are introduced into heterogeneous relations determined by different node types and edge types to achieve heterogeneous relation characteristic capture. Given the number of memory units M of a particular type of relationship, the memory-enhanced heterogeneous relationship encoder can be described by the following equation:
Figure GDA0003786233400000113
Figure GDA0003786233400000114
wherein: eta (a function of a coefficient specific to the target node and having a structure in which the target node passes through a mathematical bandOffset linearly transformed activation values, parameters, of the activated function
Figure GDA0003786233400000115
And b m All are parameters that can be learned. Memory unit of memory enhanced heterogeneous relation encoder
Figure GDA0003786233400000116
Are also mathematical parameters.
The output of the memory-enhanced heterogeneous relation encoder is obtained by the product of the sum of the output products of each memory unit and the corresponding target node specific coefficient function and the embedded vector of the given specific type relation starting point. In practice, the activation function σ () may be freely selected, and in this embodiment, taking LeakyReLU as an example, σ (x) = max (x, α x) may be selected as the activation function, where the negative slope α may take any value within the interval [0, 1).
The message fusion function Aggre (-) is also different for different types of nodes:
in a preferred embodiment, for user node u i And its social relation S corresponds to the neighbor
Figure GDA0003786233400000117
And the corresponding neighbors of the user on the user-commodity interaction history Y
Figure GDA0003786233400000118
The message fusion function can be described by the following formula:
Figure GDA0003786233400000121
wherein:
Figure GDA0003786233400000122
for user node Y i The number of social neighbors of (a) is,
Figure GDA0003786233400000123
as a user nodeu i The number of interaction histories.
For commodity node v j The message includes two parts from the user and from the category relationship, and the message fusion function can be expressed as the following formula:
Figure GDA0003786233400000124
wherein:
Figure GDA0003786233400000125
is a commodity v j R is the commodity v j And a certain category of related nodes.
The node r of the category relationship between the commodities can be described by the following formula:
Figure GDA0003786233400000126
wherein: n is a radical of hydrogen r Representing the neighbors of the class relationship node r.
Preferably, the output of the fused message is embedded into a vector, and can be optionally normalized by Layernorm and added with a self-loop to enhance the stability of the model during training. For a certain point t of the graph, this process can be expressed as the following formula:
Figure GDA0003786233400000131
wherein: w is a 1 And w 2 For learnable scaling and bias parameters, μ and σ represent the output embedding vector H, respectively (l) [t]Mean and variance of.
(3) And after the node output embedded vectors of each layer of the graph neural network are obtained, fusing the node output embedded vectors to obtain fusion node embedded vectors of the nodes.
For a certain node t of the graph, its fusion node embedding vector can be described by the following formula:
Figure GDA0003786233400000132
wherein H (0) [t]Representing the node initial embedded vector.
If Layernorm is selected not to be used in (2), the fusion process can be described using the following formula:
H * [t]＝Layernorm(H (0) [t]||H (1) [t]||…||H (l) [t])
for commodity nodes which do not need to be subjected to recalibration, the fusion nodes of the commodity nodes are embedded into vectors, namely the commodity fusion embedded vectors in the prediction stage.
And S4, recalibrating, namely recalibrating the user fusion node embedded vector obtained in the characteristic extraction stage to obtain a user final expression embedded vector.
The user fusion node embedded vector obtained by the multilayer jump connection neural network needs to be recalibrated through a user embedded vector recalibration network to obtain a user final representation embedded vector. The recalibration method is to add the user fusion node embedded vector and the convolution result of the user node containing the self-loop first-order graph in the social relation to obtain the user final representation embedded vector.
And recalibrating the embedded vector of the user fusion node, namely performing pooling operation on the user and the social relation connection node of the user, wherein recalibration output is the user final representation embedded vector.
The user fusion node embedded vector is recalibrated, so that the user embedded vector is influenced by social friends of the user more greatly, and the final recommendation effect can be improved in the actual process. User u i The recalibration result of (c) can be described by the following formula:
Figure GDA0003786233400000141
and S5, recommending and predicting, wherein preference prediction is carried out by using the embedding vector finally expressed by the user and the commodity fusion embedding vector, and a recommendation sequence is obtained.
In the recommendation prediction stage, the embedded vectors of the nodes corresponding to the target user and the target commodity are used for prediction, and the simplest inner product mode can be adopted for logic value prediction. And the commodity recommendation sequence is arranged according to the logic value descending order.
The result of the network prediction is determined by the inner product of the final expression embedded vector of the user after recalibration and the commodity fusion embedded vector, and the larger the value is, the higher the preference degree of the user to the commodity is, and vice versa. And finally, determining the result of the recommendation sequence according to the value in a descending order, wherein the commodity corresponding to the prediction result with the maximum value is the commodity with the strongest preference of the user. For the data set with negative sampling, the predicted commodity set is a union of the positive sample set and the negative sample set, and the final recommendation result is determined according to the recommendation value of the set.
Recommending a prediction stage to target user u i And target commodity v j The prediction function is defined as follows:
ξ(u i ,v j )＝τ(H * [u i ])·H * [v j ]
the larger the predicted value is, the larger the user u i For commodity v j The stronger the preference degree of (2), the recommended commodities are arranged in descending order according to the size of the predicted value.
In one embodiment, the method further comprises:
and S6, monitoring the prediction result in a heterogeneous graph neural network training stage so as to optimize network parameters.
The heterogeneous neural network training phase uses BPRLoss as a loss function to supervise and then backpropagate the gradient into the network.
Training phase, for user u i And a proof sample commodity
Figure GDA0003786233400000142
And a negative sample commodity
Figure GDA0003786233400000143
Need to outputThe predicted value of (a) is supervised training, and the loss of training can be defined as follows:
Figure GDA0003786233400000151
wherein the training data set O { (i, j) + ,j - )|(i,j + )∈Y + ,i,j - )∈Y - H, i.e. user u i And history of user-goods interactions Y occurring in the data set + Middle sample commodity
Figure GDA0003786233400000152
And history of user-merchandise interactions Y not present in the dataset - Middle load sample commodity
Figure GDA0003786233400000153
Subscripts corresponding to the constituent triplets. Theta is a training parameter, delta (-) is a sigmoid function, and lambda is a regularization adjustment factor, and a suitable value can be selected according to actual conditions.
The optimization goal is to minimize the loss of training, and the process can be optimized by calculating the gradient of the loss to each parameter and then using a back propagation algorithm to perform model optimization.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, simplifications, substitutions, and combinations which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such modifications are intended to be included in the scope of the present invention.

Claims (9)

1. A recommendation method based on a heterogeneous graph neural network is characterized by comprising the following steps:
collecting a data set with the social relationship among users, the historical data of user-commodity interaction and commodity category information, filtering invalid data and carrying out negative sampling;
randomly selecting a user set and a related commodity set from the data set, and carrying out multi-level graph sampling and graph building;
and (3) node feature extraction: inputting the constructed graph into a heterogeneous graph neural network comprising a plurality of layers of heterogeneous graph memory network layers for processing to obtain a fusion node embedded vector of nodes; for the commodity nodes which do not need to be subjected to the recalibration step, the fusion node embedded vectors of the commodity nodes are commodity fusion embedded vectors;
the processing procedure of the abnormal pattern memory network layer comprises the following steps:
1) Extracting the characteristics of an edge end point of each edge in the graph by using a memory enhanced heterogeneous relation encoder;
2) Fusing the messages received by each node to obtain an output embedded vector after the node messages are fused;
3) Directly taking the output embedded vector after fusing each node message as a node output embedded vector output by a heterogeneous graph memory network layer; or, carrying out standardization by using Layernorm and adding the output of a memory enhanced heterogeneous relation encoder using a self-loop to serve as a node output embedded vector of the final output of a heterogeneous graph memory network layer;
recalibration: recalibrating the user fusion node embedded vector to obtain a user final representation embedded vector;
and (4) recommendation and prediction: and performing preference prediction by using the user final expression embedded vector and the commodity fusion embedded vector, and obtaining a recommendation sequence.
2. The method of claim 1, wherein the multi-level graph sampling and mapping process comprises:
(1) Randomly selecting a user commodity pair of the data set, if the commodity pair is not subjected to negative sampling, further performing commodity negative sampling, and taking the user, the commodity and the negative sampling commodity in the selected set as seed nodes of graph sampling;
(2) And carrying out multi-order graph sampling on the seed nodes, determining the order and the number of sampling points of each order, and then carrying out graph sampling.
3. The recommendation method according to claim 1, wherein the heterogeneous graph neural network comprises a multi-layer hopping connection neural network and a user-embedded vector recalibration network, wherein:
the multi-layer jump connection neural network uses a multi-layer jump structure, and each layer of the network layer of the layer diagram has a jump connection to the final output;
and the user embedded vector recalibration network is used for recalibrating the user fusion node embedded vectors.
4. The recommendation method according to claim 1, wherein the step of extracting fusion node embedding vectors of nodes comprises:
(1) Inputting a multi-order graph sample and a graph constructed by graph construction into a heterogeneous graph neural network, and obtaining an initial embedded vector of a node through an initial embedded vector table;
(2) Obtaining node output embedded vectors of each layer of heterogeneous graph memory network layer of the nodes through a plurality of layers of heterogeneous graph memory network layers;
(3) And fusing the initial embedded vectors of the nodes and the node output embedded vectors of each layer of the heteromorphic image memory network layer to obtain fused node embedded vectors of the nodes.
5. The recommendation method according to claim 4, wherein the memory-enhanced heterogeneous relationship encoder comprises the operations of:
a) Obtaining the coefficient of each memory unit of the memory-enhanced heterogeneous relation encoder of the lower end point of the corresponding relation of the edges by using the bias linear transformation of the learnable band after the activation function is activated;
b) And performing linear transformation on the embedded vector of the lower starting point of the corresponding relation by using the sum of the products of all the memory units and the corresponding coefficients thereof as a transformation matrix.
6. The recommendation method according to claim 4, wherein the message fusion function for fusing the messages received by each node has different expressions in different types of nodes, and comprises:
for the user node, the message fusion function is expressed as the sum of the normalized social relationship of the user node and the messages obtained by the user-commodity interaction history through the memory enhanced heterogeneous relationship encoder;
for the commodity node, the message fusion function is expressed as the sum of the normalized user-commodity interaction history of the commodity node and the normalized commodity generic relation obtained by the memory enhanced heterogeneous relation encoder;
for the commodity category node, the message fusion function is expressed as the sum of the messages of the normalized commodity category node commodity generic relationship obtained by the memory enhanced heterogeneous relationship encoder.
7. The recommendation method according to claim 4, wherein the method of obtaining the fusion node embedding vector of the nodes comprises: and splicing the initial embedded vectors of the nodes with the output of each layer of the heterogeneous graph memory network layer, and standardizing by using Layernorm.
8. The recommendation method according to claim 1, wherein the recalibration method comprises: and adding the user fusion node embedded vector and the result of the self-looping first-order graph convolution of the user node on the social relationship to obtain a user final representation embedded vector.
9. The recommendation method according to claim 1, further comprising:
the heterogeneous graph neural network training phase uses BPRLoss as a loss function to supervise and then backpropagate the gradient into the heterogeneous graph neural network.
CN202110296182.1A 2021-03-19 2021-03-19 Recommendation method based on heterogeneous graph neural network Active CN112990972B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110296182.1A CN112990972B (en) 2021-03-19 2021-03-19 Recommendation method based on heterogeneous graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110296182.1A CN112990972B (en) 2021-03-19 2021-03-19 Recommendation method based on heterogeneous graph neural network

Publications (2)

Publication Number Publication Date
CN112990972A CN112990972A (en) 2021-06-18
CN112990972B true CN112990972B (en) 2022-11-18

Family

ID=76333501

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110296182.1A Active CN112990972B (en) 2021-03-19 2021-03-19 Recommendation method based on heterogeneous graph neural network

Country Status (1)

Country Link
CN (1) CN112990972B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113409121B (en) * 2021-06-29 2022-02-15 南京财经大学 Cross-border e-commerce recommendation method based on heterogeneous graph expression learning
CN113392334B (en) * 2021-06-29 2024-03-08 长沙理工大学 False comment detection method in cold start environment
CN113609398B (en) * 2021-08-17 2023-09-19 石家庄铁道大学 Social recommendation method based on heterogeneous graph neural network
CN113849725B (en) * 2021-08-19 2022-03-29 齐鲁工业大学 Socialized recommendation method and system based on graph attention confrontation network
CN114417161B (en) * 2022-01-21 2023-04-07 杭州碧游信息技术有限公司 Virtual article time sequence recommendation method, device, medium and equipment based on special-purpose map
CN115082142B (en) * 2022-05-10 2024-04-30 华南理工大学 Recommendation method, device and medium based on heterogeneous relational graph neural network
CN115099886B (en) * 2022-05-25 2024-04-19 华南理工大学 Long-short interest sequence recommendation method, device and storage medium
CN115082147B (en) * 2022-06-14 2024-04-19 华南理工大学 Sequence recommendation method and device based on hypergraph neural network
CN116662676A (en) * 2023-06-09 2023-08-29 北京华品博睿网络技术有限公司 Online recruitment bidirectional reciprocity recommendation system and method based on multi-behavior modeling
CN117390266B (en) * 2023-10-08 2024-04-30 宁夏大学 Project recommendation method based on high-order neighbor generation algorithm and heterogeneous graph neural network

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9760609B2 (en) * 2013-11-22 2017-09-12 Here Global B.V. Graph-based recommendations service systems and methods
US11287894B2 (en) * 2018-03-09 2022-03-29 Adobe Inc. Utilizing a touchpoint attribution attention neural network to identify significant touchpoints and measure touchpoint contribution in multichannel, multi-touch digital content campaigns
CN110162690B (en) * 2018-10-23 2023-04-18 腾讯科技（深圳）有限公司 Method and device for determining interest degree of user in item, equipment and storage medium
CN110610715B (en) * 2019-07-29 2022-02-22 西安工程大学 Noise reduction method based on CNN-DNN hybrid neural network
CN111428147B (en) * 2020-03-25 2021-07-27 合肥工业大学 Social recommendation method of heterogeneous graph volume network combining social and interest information
CN111523047B (en) * 2020-04-13 2022-08-09 中南大学 Multi-relation collaborative filtering algorithm based on graph neural network

Also Published As

Publication number Publication date
CN112990972A (en) 2021-06-18

Similar Documents

Publication Publication Date Title
CN112990972B (en) Recommendation method based on heterogeneous graph neural network
CN111797321B (en) Personalized knowledge recommendation method and system for different scenes
CN110717098B (en) Meta-path-based context-aware user modeling method and sequence recommendation method
Lima et al. Domain knowledge integration in data mining using decision tables: case studies in churn prediction
CN114817663B (en) Service modeling and recommendation method based on class perception graph neural network
CN115082142A (en) Recommendation method, device and medium based on heterogeneous relational graph neural network
CN112364242B (en) Graph convolution recommendation system for context awareness
CN113761359B (en) Data packet recommendation method, device, electronic equipment and storage medium
Alhamdani et al. Recommender system for global terrorist database based on deep learning
CN111797320A (en) Data processing method, device, equipment and storage medium
CN116308685A (en) Product recommendation method and system based on aspect emotion prediction and collaborative filtering
CN116703509A (en) Online shopping assistant construction method for live marketing commodity quality perception analysis
CN117077735A (en) Dimension-dependent integrated service quality prediction method based on convolutional neural network
CN111143700B (en) Activity recommendation method, activity recommendation device, server and computer storage medium
CN114757391B (en) Network data space design and application method oriented to service quality prediction
CN116010696A (en) News recommendation method, system and medium integrating knowledge graph and long-term interest of user
CN115618079A (en) Session recommendation method, device, electronic equipment and storage medium
CN114897607A (en) Data processing method and device for product resources, electronic equipment and storage medium
CN115080856A (en) Recommendation method and device and training method and device of recommendation model
CN112948681B (en) Multi-dimensional feature fused time series data recommendation method
CN113254788A (en) Big data based recommendation method and system and readable storage medium
CN117633371B (en) Recommendation method, device and readable storage medium based on multi-attention mechanism
EP4156046A1 (en) Operation prediction device, model training method for same, and operation prediction method
CN117786234B (en) Multimode resource recommendation method based on two-stage comparison learning
CN116628310A (en) Content recommendation method, device, equipment, medium and computer program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant