CN116894476A - Multi-behavior attention self-supervision learning method based on double channels - Google Patents

Multi-behavior attention self-supervision learning method based on double channels Download PDF

Info

Publication number
CN116894476A
CN116894476A CN202310823335.2A CN202310823335A CN116894476A CN 116894476 A CN116894476 A CN 116894476A CN 202310823335 A CN202310823335 A CN 202310823335A CN 116894476 A CN116894476 A CN 116894476A
Authority
CN
China
Prior art keywords
behavior
user
graph
matrix
sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310823335.2A
Other languages
Chinese (zh)
Inventor
王楠
曲明月
钟颖莉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Heilongjiang University
Original Assignee
Heilongjiang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Heilongjiang University filed Critical Heilongjiang University
Priority to CN202310823335.2A priority Critical patent/CN116894476A/en
Publication of CN116894476A publication Critical patent/CN116894476A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a multi-behavior attention self-supervision learning method based on double channels. The method is capable of distinguishing between different user behavior perceived preferences. Different behaviors of user interaction with the item are exploited. The three self-supervision learning modes of the application not only enhance the double-channel characterization result, but also enable the model to obtain more auxiliary supervision signals in the self-supervision learning in the channels and among the channels, thereby effectively relieving the problem of sparse supervision signals.

Description

Multi-behavior attention self-supervision learning method based on double channels
Technical Field
The application relates to the technical field of multiple behavior preference prediction based on dual-channel cross-behavior dependence modeling, in particular to a bidirectional coding representation converter, a graph neural network and a personalized behavior recommendation method for self-supervision learning. In particular to a multi-behavior attention self-supervision learning method based on double channels.
Background
In recent years, with the development of the mobile internet, the center of gravity of electronic commerce is shifted from a personal computer to a smart phone, various mobile electronic commerce platforms have been developed, a training network based on user interaction has also been developed, and in particular, multi-behavior recommendation has been widely focused on in academia. The massive interactive data generated by millions of users provides excellent opportunities for exploring potential intentions in various behaviors of the users, but meanwhile, the massive data also causes the users to fall into the dilemma of sparse behavior information types, and personalized recommendation cannot be effectively performed. To solve this problem, multi-behavior recommendation has become one of the most popular research subjects in the field of e-commerce platform mining, as an effective method for helping users not only to explore their own interesting commodities, but also to offer an e-commerce platform to attract more potential users.
Often, intent recommendations can save the user's time without any input, as typing on mobile devices is more difficult than on desktop computers, which will improve the user's liveness and shopping experience, and the user can share their belongings with friends through sharing behaviors that contain user context information, which can generally reflect the user's interests. By deep mining of these behavioral information, the preferences of deeper users for the merchandise can be revealed. The interactive behavior of the user represents the interaction of the user with the commodity and also shows commodity characteristics and user characteristics. From the perspective of a user, the user clicks the commodity, adds a shopping cart to the commodity, and finally clears a plurality of commodities in the shopping cart together; from a merchandise perspective, for merchandise including a buy-in, a favorite interactive action is more attractive to the user than merchandise with only a single interactive action. In addition, users typically share items with friends. These unique behaviors make the multi-behavior recommendation different from the conventional recommendation system, so that it is necessary to fully understand the interaction information of the user in the multi-behavior with respect to the commodity, thereby developing a new algorithm for multi-behavior recommendation.
Disclosure of Invention
The application aims to solve the problems in the prior art, and provides a multi-behavior attention self-supervision learning method based on double channels.
The application is realized by the following technical scheme, and provides a multi-behavior attention self-supervision learning method based on double channels, which comprises the following steps:
step one, acquiring a commodity interaction data set from a cat and CIKM2019 electronic commerce artificial intelligence challenge game, and selecting T% of data as a training data set and (1-T%) of data as a test data set, wherein the training data set comprises various interaction behavior histories of users on commodities, users and users;
step two, the user set in the training data set is U, and U= { U 1 ，u 2 ，...，u q ， . ..，u N E { 1..N }, where u q The q-th user, N is the number of users; commodity set is I, I= { I 1 ，i 2 ，...，i t ，...，i T T e { 1.,.. T }, where i t T is the T user, and T is the number of commodities; the behavior set is B, b= { B 1 ，b 2 ，...，b k，. ...，i K K e { 1.,.. K }, wherein b k K is the kth behavior, K is the number of behaviors;
step three, constructing a user-commodity interaction sequence according to the behavior history of the user from the aspect of the sequence channel;
step four, from the view of a sequence channel, a BERT4Rec calculation method is introduced because the depth bidirectional model is better than the unidirectional modelGELU is gaussian error linear unit activation function; w represents the weight matrix of the GELU activation function, and b represents the bias; />The softmax is used as an output activation function, the normalization operation is carried out on the spliced results of various behavior sequences, and for different users, the users have different behavior interaction sequences to cause different coding results;
step five, considering from a sequence channel, designing self-supervision loss according to the characterization result;
step six, taking the graph channel into consideration, acquiring enough available information of the user; the user has various behaviors including clicking, adding to a shopping cart, collecting and purchasing; definition g= (V, E), V represents that the node set contains a user set U E U and a project set I E I, i.e. (U, I) E V; e represents different interaction behaviors between the user node and the project node; the embedding of the multi-behavior graph is composed of a plurality of behavior sub-graph embeddings, and the behavior sub-graph embeddings are expressed as G b ＝(V b ，E b )；
Step seven, considering the graph channel, assisting the behavior graphAnd target behavior diagram->As an input of attention;
step eight, considering the graph channel, designing self-supervision learning of a multi-behavior interaction graph in the channel, and enhancing various behavior data supervision signals through the self-supervision learning;
and step nine, from the consideration of a sequence channel and a graph channel, enhancing a supervision signal through self-supervision learning combined by two channels.
Further, in step three, each element in the user interaction behavior sequence is set as a feature vector of a tripletRepresenting user q interacting with item x with a kth behavior; the multi-behavior sequence of the user contains interaction information of a single user, and the multi-behavior interaction sequence of the user is mapped into an initial embedded feature matrix>It contains the merchandise that user q interacts through all actions; feature vector of auxiliary behavior interaction sequence +.>Feature vector of interaction sequence with target behavior +.>As a multi-behavior interaction sequence depends on the input of the encoder.
Further, the feature vector of each auxiliary behavior and the feature vector of the target behavior are calculated, wherein the calculation process is that Wherein W is Q ，W Q ∈R d*n A weight matrix that is a learnable behavior vector; />Representation->Is a transpose of (2); />Representing an association matrix between the auxiliary behavior k and the target behavior k'; /> Each association matrix->The attention score +.about.of the interval conforming to the probability distribution value can be obtained through softmax normalization>The softmax obtains the behavior most similar to the purchasing behavior by calculating cosine similarity;W V ∈R d*n is a weight matrix of a learnable behavior vector.
Further, in step five, different behaviors of the same user are treated as positive sample pairsDifferent behaviors between different users as negative sample pairs +.>Whereby the self-supervising penalty on the user behavior is: />Wherein->In (a) and (b)Representing the computed cosine similarity.
Further, in step six, graph convolution is used to learn node characterization of the graph, aggregate and pass node characteristics; the process of graph convolution is specifically: embedding adjacency matrix A for each behavior subgraph k It is formed by matrix R k The specific process is as follows:each behavior subgraph is embedded into an adjacency matrix A k As an input to the normalized laplace matrix of behavior, the normalization process is: />Wherein->Degree matrix characterizing k behavior, I k Identity matrix representing k rows>The output of the graph convolution passes through a threshold function sigmoid: /> Wherein->Is the node characteristic matrix of the layer I of the nodes in the graph, W k Is a transformation matrix for behavior view information transfer; the graph convolves L layers, L represents the acquired L-order neighbor nodes, the information aggregation process is obtained through node information, the characteristics of the nodes about k behaviors in the graph are acquired, and multi-behavior context information can be stored.
Further, in step seven, the process of distinguishing the influence intensity of the auxiliary behavior pattern on the target behavior pattern through the attention is as follows:wherein W is Q ∈R d*n And W is K ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>is a matrix of attention-related coefficients; />Attention calculating procedure and +.>Is considered as weight multiplication auxiliary behavior>Wherein W is V ∈R d*n Is a weight matrix of a behavior matrix that can be iteratively updated continuously,/>the auxiliary behavior feature matrix of the target behavior is used as the final output of the cross-behavior interaction diagram attention coder.
Further, in step eight, different behavioral views of the same user are treated as positive sample pairsDifferent behavior of different users is regarded as negative example pair +.>Maximizing mutual information between users by self-supervising positive and negative sample pairs:the consistency of the two behavior views and the difference between different user behaviors are maximized, and the behavior data supervision signals are enhanced.
Further, in step nine, the sequence channel and the view channel of the same user are regarded as positive samples, usedA representation; the sequence channels and view channels of different users are regarded as negative samples, with +.> A representation; self-supervision loss: />τ is a temperature coefficient that balances the intensity of learning between the two channels; all self-supervising losses and as final target losses: l (L) CL ＝L SCL +L GCL +L SGCL ；L SCL Is the self-monitoring of multi-behavior interaction sequences in a sequence channelLoss supervision; l (L) GCL Is the self-supervision loss of the multi-behavior interaction graph in the graph channel; final list of loss functions L CL By the sequence loss function L of each pair of actions ScL And view loss function L GCL And a sequence view loss function L SGCL The composition is formed.
The application provides an electronic device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the multi-behavior attention self-supervision learning method based on two channels when executing the computer program.
The present application proposes a computer readable storage medium for storing computer instructions which, when executed by a processor, implement the steps of the dual channel-based multi-behavioral attention self-supervised learning method.
The application has the following beneficial effects:
the application provides a multi-behavior attention self-supervision learning method based on double channels, which can distinguish the abilities of different user behaviors to perceive preferences. Different behaviors of user interaction with the project under different modes are utilized. The commonality of the multiple behaviors of the user is captured, and the three self-supervision learning modes not only enhance the double-channel characterization result, but also enable the model to obtain more auxiliary supervision signals in the self-supervision learning in the channels and among the channels, thereby effectively relieving the problem of sparse supervision signals.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is an overall schematic diagram of a dual-channel-based multi-behavior attention self-monitoring learning method according to the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Personalized recommendations are an important component in an e-commerce platform. The recommendation system enhances collaborative filtering through a neural network to achieve accurate capturing of user preferences, thereby achieving better recommendation performance. Traditional recommendation methods focus on the results of a user's single behavior, ignoring modeling that utilizes the user's multiple interactions (clicking, joining shopping cart, purchasing). Although much research has focused on multi-behavioral modeling, there are still two important challenges: 1) There are still challenges in identifying multimodal relationships of behaviors due to the omission of multiple behavior context information. 2) The supervisory signals remain sparse. In order to solve the problem, the application provides a multi-behavior attention double-channel contrast learning method, multiple behavior context information is extracted from different types of interactions of a user through set self-supervision learning, behavior dependence is provided for the user, and multiple relations of different behaviors are obtained. The robustness of the model is enhanced. The application designs a double-channel self-supervision learning to enhance a data supervision signal. A large number of experiments on two real data sets show that the method of the application is always superior to the most advanced multi-behavior recommendation method.
With reference to fig. 1, the application provides a multi-behavior attention self-supervision learning method based on two channels, which comprises the following steps:
step one, acquiring a commodity interaction data set from a cat and CIKM2019 electronic commerce artificial intelligence challenge game, and selecting T% of data as a training data set and (1-T%) of data as a test data set, wherein the training data set comprises various interaction behavior histories of users on commodities, users and users;
step two, the user set in the training data set is U, and U= { U 1 ，u 2 ，...，u q ， . ..，u N E { 1..N }, where u q The q-th user, N is the number of users; commodity set is I, I= { I 1 ，i 2 ，...，i t ，....，i T T e { 1.,.. T }, where i t T is the T user, and T is the number of commodities; the behavior set is B, b= { B 1 ，b 2 ，...，b k ，....，i K K e { 1.,.. K }, wherein b k K is the kth behavior, K is the number of behaviors;
step three, constructing a user-commodity interaction sequence according to the behavior history of the user from the aspect of the sequence channel;
step four, from the view of a sequence channel, a BERT4Rec calculation method is introduced because the depth bidirectional model is better than the unidirectional modelGELU is gaussian error linear unit activation function; w represents the weight matrix of the GELU activation function, and b represents the bias; />The softmax is used as an output activation function, the normalization operation is carried out on the spliced results of various behavior sequences, and for different users, the users have different behavior interaction sequences to cause different coding results;
step five, considering from a sequence channel, designing self-supervision loss according to the characterization result;
step six, taking the graph channel into consideration, acquiring enough available information of the user; the user has various behaviors including clicking, adding to a shopping cart, collecting and purchasing; definition g= (V, E), V represents that the node set contains a user set U E U and a project set I E I, i.e. (U, I) E V; e represents different interaction behaviors between the user node and the project node; the embedding of the multi-behavior graph is composed of a plurality of behavior sub-graph embeddings, and the behavior sub-graph embeddings are expressed as G b ＝(V b ，E b )；
Step seven, considering the graph channel, assisting the behavior graphAnd target behavior diagram->As an input of attention;
step eight, considering the graph channel, designing self-supervision learning of a multi-behavior interaction graph in the channel, and enhancing various behavior data supervision signals through the self-supervision learning;
and step nine, from the consideration of a sequence channel and a graph channel, enhancing a supervision signal through self-supervision learning combined by two channels.
In step three, each element in the user interaction behavior sequence is set as a feature vector of a triplet Representing user q interacting with item x with a kth behavior; the multi-behavior sequence of the user contains interaction information of a single user, and the multi-behavior interaction sequence of the user is mapped into an initial embedded feature matrixIt contains the merchandise that user q interacts through all actions; feature vector of auxiliary behavior interaction sequence +.>Feature vector of interaction sequence with target behavior +.>As a multi-behavior interaction sequence depends on the input of the encoder.
Calculating the characteristic vector of each auxiliary behavior and the characteristic vector of the target behavior by the following calculation process Wherein WQ, WQ e Rd n is a weight matrix of the learnable behavior vector; />Representation ofIs a transpose of (2); />Representing an association matrix between the auxiliary behavior k and the target behavior k'; /> Each association matrix->The attention score +.about.of the interval conforming to the probability distribution value can be obtained through softmax normalization>The softmax obtains the behavior most similar to the purchasing behavior by calculating cosine similarity;WV e Rd n is a weight matrix of learnable behavior vectors.
In step five, different roles of the same user are treated as positive sample pairsDifferent behaviors between different users as negative sample pairs +.>Thereby closingThe self-supervision loss of user behavior is:wherein->Is-> Representing the computed cosine similarity.
In step six, graph convolution is used for learning node characterization of the graph, aggregating and transmitting node characteristics; the process of graph convolution is specifically: embedding adjacency matrix A for each behavior subgraph k It is formed by matrix R k The specific process is as follows: each behavior subgraph is embedded into an adjacency matrix A k As an input to the normalized laplace matrix of behavior, the normalization process is: />Wherein->Degree matrix characterizing k behavior, I k Identity matrix representing k rowsThe output of the graph convolution passes through a threshold function sigmoid: />Wherein->Is the node characteristic matrix of the layer I of the nodes in the graph, W k Is a transformation matrix for behavior view information transfer; the graph convolves L layers, L represents the acquired L-order neighbor nodes, the information aggregation process is obtained through node information, the characteristics of the nodes about k behaviors in the graph are acquired, and multi-behavior context information can be stored.
In the seventh step, the process of distinguishing the influence intensity of the auxiliary behavior diagram on the target behavior diagram through the attention is as follows: wherein W is Q ∈R d*n And W is K ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>is a matrix of attention-related coefficients; />Attention calculation process of (a)Is considered as weight multiplication auxiliary behavior>Wherein W is V ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>the auxiliary behavior feature matrix of the target behavior is used as the final output of the cross-behavior interaction diagram attention coder.
In step eight, different behavioral views of the same user are treated as positive sample pairsDifferent behavior of different users is regarded as negative example pair +.>Maximizing mutual information between users by self-supervising positive and negative sample pairs: />The consistency of the two behavior views and the difference between different user behaviors are maximized, and the behavior data supervision signals are enhanced.
In step nine, the sequence channel and the view channel of the same user are regarded as positive samples, usedA representation; the sequence channels and view channels of different users are regarded as negative samples, with +.>A representation; self-supervision loss: />τ is a temperature coefficient that balances the intensity of learning between the two channels; all self-supervising losses and as final target losses: l (L) CL ＝L SCL +L GCL +L SGCL ；L SCL Is the self-supervision loss of the multi-behavior interaction sequence in the sequence channel; l (L) GCL Is the self-supervision loss of the multi-behavior interaction graph in the graph channel; final list of loss functions L CL By the sequence loss function L of each pair of actions SCL And view loss function L GCL And a sequence view loss function L SGCL The composition is formed.
Examples
The application provides a universal and flexible multi-behavior relation learning framework, namely a multi-behavior attention self-supervision learning method based on double channels. In particular, the method first proposes a multi-behavior-dependent encoder that learns the interdependencies of behaviors by incorporating behavior representations of a particular type in different types of user-project interactions. Then, the problem of data sparseness is solved by double-channel multi-behavior self-supervision learning. To model the multi-type behavior pattern dependencies, comprehensive learning is performed to make recommendations. The dual-channel multi-behavior-dependent self-supervision learning model designed by the application parameterizes each type of user-project interaction into independent dependency representation of the individual behavior types of the learning user in an embedded space, and enhances the data supervision signals by utilizing the self-supervision learning paradigm among channels.
The application provides a multi-behavior attention self-supervision learning method based on double channels, which comprises the following steps:
step 1, acquiring a commodity interaction data set from a daily cat and CIKM2019 electronic commerce artificial intelligent challenge game, wherein the data of T% is selected as training data, and the data of (1-T%) is used as test data, and the training data set comprises various interaction behavior histories of users on commodities, users and users;
step 2, the user set in the training set is U, u= { U 1 ，u 2 ，...，u q ，...，u. N E { 1..N }, where u q For the q-th user, N is the number of users. Commodity set is I, I= { I 1 ，i 2 ，...，i t ，....，i T T e { 1.,.. T }, where i t For the T-th user, T is the number of commodities. The behavior set is B, b= { B 1 ，b 2 ，...，b k ， . ...，i K K e { 1.,.. K }, wherein b k For the kth behavior, K is the number of behaviors.
And 3, constructing a user-commodity interaction sequence according to the behavior history of the user from the aspect of the sequence channel. Setting each element in the user interaction behavior sequence as a feature vector of a triplet Representing user q interacting with item x with the kth behavior. The multi-behavioral sequence of users contains interactive information for a single user. Whereby the user's multi-behavioral interaction sequence is mapped to an initial embedding forming feature matrix +.> Which contains the merchandise that user q interacts with through all actions. Feature vector of auxiliary behavior interaction sequence +.>Feature vector of interaction sequence with target behavior +.>As the input of the multi-behavior interaction sequence dependent encoder, the calculation method of the multi-behavior interaction sequence dependent encoder is consistent with the calculation method of attention, the characteristic vector of each auxiliary behavior and the characteristic vector of the target behavior are calculated, and the calculation process is-> Wherein W is Q ，W Q ∈R d*n Is a weight matrix of a learnable behavior vector. />Representation->Is a transpose of (a). />Representing the correlation matrix between the auxiliary behavior k and the target behavior k'. /> Each association matrix->The attention score +.about.of the interval conforming to the probability distribution value can be obtained through softmax normalization>softmax derives the most similar behavior to the purchase behavior by computing cosine similarity.W V ∈R d*n Is a weight matrix of a learnable behavior vector. To prevent the over-fitting problem while avoiding excessive computation time costs, we use dropout to get +.>
Step 4, from the view of a sequence channel, introducing a computational square note of BERT4Rec because the depth bidirectional model is better than the unidirectional modelGel is a gaussian error linear element activation function. W represents the weight matrix of the GELU activation function and b represents the bias. />And taking softmax as an output activation function, and normalizing the spliced results of various behavior sequences. For different users, they have different behavior interaction sequences resulting in different encoding results.
And 5, considering the sequence channel, and designing self-supervision loss aiming at the characterization result. Specifically, different acts of the same user are treated as positive sample pairsDifferent behavior between different users as negative sample pairsThe self-supervising penalty on user behavior is therefore: /> Wherein->Is->Representing the computed cosine similarity.
And 6, considering the graph channel, acquiring enough available information by using single information of the user. As users have a variety of behaviors, including clicking, joining shopping carts, collecting and purchasing behaviors. Definition g= (V, E), V denotes that the node set contains a user set U E U and a project set I E I, i.e. (U, I) E V. E represents different interaction behaviors between the user node and the project node. Furthermore, the embedding of the multiple behavior graph is made up of multiple behavior subgraph embeddings, so the behavior subgraph embeddings can be represented as G b ＝(V b ，E b ). Behavior subgraphs, G, e.g., made up of items clicked by a user click ＝(V click ，E click ) Wherein G is click Item view representation representing user interaction through click behavior, V click Representing user and project nodes connected with click behavior, E click Representing the clicking behavior of the user. First, graph convolution is directed to learning node characterizations of graphs, aggregating and delivering node features. The process of graph convolution, in particular, embedding adjacency matrix A for each behavior subgraph k It is formed by matrix R k The specific process comprises the following steps:each behavior subgraph is embedded into an adjacency matrix A k As an input to the normalized laplace matrix of behavior, the normalization process is as follows: />Wherein->Degree matrix characterizing k behavior, I k Identity matrix representing k rowsBecause the graph convolution can well acquire the higher-order dependency relationship of the user nodes, the graph convolution can be used for better acquiring the global characterization of all the user nodes for the multi-behavior interaction graph of the user. The output method of the graph convolution goes through a threshold function +.>Wherein->Is the node characteristic matrix of the layer I of the nodes in the graph, W k Is a transformation matrix for behavioral view information delivery. The graph convolves L layers, L represents the obtained L-order neighbor nodes, the information aggregation process is obtained through node information, the characteristics of the nodes about k behaviors in the graph are obtained, and multi-behavior context information can be stored.
Step 7, considering the graph channel, assisting the behavior graphAnd target behavior diagram->As an input of attention. The strength of the effect of the auxiliary behavior diagram on the target behavior diagram is distinguished through the attention, such as: />Wherein W is Q ∈R d*n And W is K ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>is a matrix of attention-related coefficients.The attention calculating process of (2) is regarded as weight multiplication assisting action as the same as the attention calculating process in step 3 +.>Wherein W is V ∈R d*n Is a weight matrix of the behavior matrix that can be iteratively updated continuously. />The auxiliary behavior feature matrix of the target behavior is used as the final output of the cross-behavior interaction diagram attention coder.
And 8, considering the graph channel, designing self-supervision learning of the multi-behavior interaction graph in the channel, and enhancing various behavior data supervision signals through the self-supervision learning. Different behavioral views of the same user are considered as positive sample pairsDifferent behavior of different users is regarded as negative example pair +.>Maximizing mutual information between users by self-supervising positive and negative sample pairs:consistency of the two behavior views. And the difference between different user behaviors is maximized, so that the behavior data supervision signals are enhanced.
Step 9 from the sequence channel and the map channel point of view,the self-supervision learning combined by the double channels is more beneficial to enhancing supervision signals. The sequence channel and the view channel of the same user are regarded as positive samples, usedAnd (3) representing. The sequence channels and view channels of different users are regarded as negative samples, with +.>And (3) representing. Self-supervision loss:τ is the temperature coefficient, balancing the intensity of learning between the two channels. All self-supervising losses and as final target losses: l (L) CL ＝L SCL +L GCL +L SaCL 。L SCL Is the loss of self-supervision of the sequence of interactions within the sequence channels mentioned in step 5. L (L) GCL Is the self-supervision penalty of the multi-behavior interaction graph in the graph channel mentioned in step 9 section. The present application thus complements the two proposed self-supervising losses. Final list of loss functions L CL By the sequence loss function L of each pair of actions SCL And view loss function L GCL And a sequence view loss function L SGCL The composition is formed.
The application provides an electronic device, which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the multi-behavior attention self-supervision learning method based on two channels when executing the computer program.
The present application proposes a computer readable storage medium for storing computer instructions which, when executed by a processor, implement the steps of the dual channel-based multi-behavioral attention self-supervised learning method.
The memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a Read Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and direct memory bus RAM (DR RAM). It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in the processor for execution. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method. To avoid repetition, a detailed description is not provided herein.
It should be noted that the processor in the embodiments of the present application may be an integrated circuit chip with signal processing capability. In implementation, the steps of the above method embodiments may be implemented by integrated logic circuits of hardware in a processor or instructions in software form. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
The application has been described in detail with respect to a dual-channel-based multi-behavior attention self-supervising learning method, and specific examples are applied herein to illustrate the principles and embodiments of the application, and the above examples are only for aiding in understanding the method and core ideas of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims (10)

1. A multi-behavior attention self-supervision learning method based on double channels is characterized in that: the method comprises the following steps:
step one, acquiring a commodity interaction data set from a cat and CIKM2019 electronic commerce artificial intelligence challenge game, and selecting T% of data as a training data set and (1-T%) of data as a test data set, wherein the training data set comprises various interaction behavior histories of users on commodities, users and users;
step two, the user set in the training data set is U, and U= { U 1 ,u 2 ,...,u q ,...,u N E { 1..N }, where u q The q-th user, N is the number of users; commodity set is I, I= { I 1 ,i 2 ,...,i t ,....,i T T e { 1.,.. T }, where i t T is the T user, and T is the number of commodities; the behavior set is B, b= { B 1 ,b 2 ,...,b k ,....,i K K e { 1.,.. K }, wherein b k K is the kth behavior, K is the number of behaviors;
step three, constructing a user-commodity interaction sequence according to the behavior history of the user from the aspect of the sequence channel;
step four, from the view of a sequence channel, a BERT4Rec calculation method is introduced because the depth bidirectional model is better than the unidirectional modelGELU is gaussian error linear unit activation function; w represents the weight matrix of the GELU activation function, and b represents the bias; />The softmax is used as an output activation function, the normalization operation is carried out on the spliced results of various behavior sequences, and for different users, the users have different behavior interaction sequences to cause different coding results;
step five, considering from a sequence channel, designing self-supervision loss according to the characterization result;
step six, taking the graph channel into consideration, acquiring enough available information of the user; the user has various behaviors including clicking, adding to a shopping cart, collecting and purchasing; definition g= (V, E), V represents that the node set contains a user set U E U and a project set I E I, i.e. (U, I) E V; e represents different interaction behaviors between the user node and the project node; the embedding of the multi-behavior graph is composed of a plurality of behavior sub-graph embeddings, and the behavior sub-graph embeddings are expressed as G b ＝(V b ,E b )；
Step seven, considering the graph channel, assisting the behavior graphAnd target behavior diagram->As an input of attention;
step eight, considering the graph channel, designing self-supervision learning of a multi-behavior interaction graph in the channel, and enhancing various behavior data supervision signals through the self-supervision learning;
and step nine, from the consideration of a sequence channel and a graph channel, enhancing a supervision signal through self-supervision learning combined by two channels.
2. The method according to claim 1, characterized in thatThe method comprises the following steps: in step three, each element in the user interaction behavior sequence is set as a feature vector of a tripletRepresenting user q interacting with item x with a kth behavior; the multi-behavior sequence of the user contains interaction information of a single user, and the multi-behavior interaction sequence of the user is mapped into an initial embedded feature matrix>It contains the merchandise that user q interacts through all actions; feature vector of auxiliary behavior interaction sequence +.>Feature vector of interaction sequence with target behavior +.>As a multi-behavior interaction sequence depends on the input of the encoder.
3. The method according to claim 2, characterized in that: calculating the characteristic vector of each auxiliary behavior and the characteristic vector of the target behavior by the following calculation processWherein W is Q ，W Q ∈R d*n A weight matrix that is a learnable behavior vector; />Representation->Is a transpose of (2); />Representing auxiliary behavior k and target behaviorAn association matrix between k'; />Each association matrix->The attention score +.about.of the interval conforming to the probability distribution value can be obtained through softmax normalization>The softmax obtains the behavior most similar to the purchasing behavior by calculating cosine similarity; />W V ∈R d*n Is a weight matrix of a learnable behavior vector.
4. A method according to claim 3, characterized in that: in step five, different roles of the same user are treated as positive sample pairsDifferent behaviors between different users as negative sample pairs +.>U, q+.p }; whereby the self-supervising penalty on the user behavior is: /> Wherein->Is->Representing the computed cosine similarity.
5. The method according to claim 4, wherein: in step six, graph convolution is used for learning node characterization of the graph, aggregating and transmitting node characteristics; the process of graph convolution is specifically: embedding adjacency matrix A for each behavior subgraph k It is formed by matrix R k The specific process is as follows:each behavior subgraph is embedded into an adjacency matrix A k As an input to the normalized laplace matrix of behavior, the normalization process is: />Wherein->Degree matrix characterizing k behavior, I k Identity matrix representing k rows>The output of the graph convolution passes through a threshold functionWherein->Is the node characteristic matrix of the layer I of the nodes in the graph, W k Is a transformation matrix for behavior view information transfer; the graph convolves L layers, L represents the acquired L-order neighbor nodes, the information aggregation process is obtained through node information, the characteristics of the nodes about k behaviors in the graph are acquired, and multi-behavior context information can be stored.
6. The method according to claim 5, wherein:in the seventh step, the process of distinguishing the influence intensity of the auxiliary behavior diagram on the target behavior diagram through the attention is as follows:wherein W is Q ∈R d*n And W is K ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>is a matrix of attention-related coefficients; /> Attention calculating procedure and +.>Is considered as weight multiplication auxiliary behavior>Wherein W is V ∈R d*n Weight matrix, which is a behavior matrix that can be updated iteratively and continuously,>the auxiliary behavior feature matrix of the target behavior is used as the final output of the cross-behavior interaction diagram attention coder.
7. The method according to claim 6, wherein: in step eight, different behavioral views of the same user are treated as positive sample pairsDifferent behavior of different users is regarded as negative example pair +.> Maximizing mutual information between users by self-supervising positive and negative sample pairs: /> The consistency of the two behavior views and the difference between different user behaviors are maximized, and the behavior data supervision signals are enhanced.
8. The method according to claim 7, wherein: in step nine, the sequence channel and the view channel of the same user are regarded as positive samples, usedA representation; the sequence channels and view channels of different users are regarded as negative samples, with +.>A representation; self-supervision loss: /> τ is a temperature coefficient that balances the intensity of learning between the two channels; all self-supervising losses and as final target losses: l (L) CL ＝L SCL +L GCL +L SGCL ；L SCL Is multiple in sequence channelSelf-supervision loss of the behavior interaction sequence; l (L) GCL Is the self-supervision loss of the multi-behavior interaction graph in the graph channel; final list of loss functions L CL By the sequence loss function L of each pair of actions SCL And view loss function L GCL And a sequence view loss function L SGCL The composition is formed.
9. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-8 when the computer program is executed.
10. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the method of any one of claims 1-8.
CN202310823335.2A 2023-07-06 2023-07-06 Multi-behavior attention self-supervision learning method based on double channels Pending CN116894476A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310823335.2A CN116894476A (en) 2023-07-06 2023-07-06 Multi-behavior attention self-supervision learning method based on double channels

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310823335.2A CN116894476A (en) 2023-07-06 2023-07-06 Multi-behavior attention self-supervision learning method based on double channels

Publications (1)

Publication Number Publication Date
CN116894476A true CN116894476A (en) 2023-10-17

Family

ID=88313109

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310823335.2A Pending CN116894476A (en) 2023-07-06 2023-07-06 Multi-behavior attention self-supervision learning method based on double channels

Country Status (1)

Country Link
CN (1) CN116894476A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117574844A (en) * 2023-11-23 2024-02-20 华南理工大学 Self-supervision learning DTCO process parameter performance specification feedback method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117574844A (en) * 2023-11-23 2024-02-20 华南理工大学 Self-supervision learning DTCO process parameter performance specification feedback method

Similar Documents

Publication Publication Date Title
CN111931062B (en) Training method and related device of information recommendation model
CN109902706B (en) Recommendation method and device
CN112529168B (en) GCN-based attribute multilayer network representation learning method
Li et al. Deep probabilistic matrix factorization framework for online collaborative filtering
Arya et al. Exploiting relational information in social networks using geometric deep learning on hypergraphs
CN111563192B (en) Entity alignment method, device, electronic equipment and storage medium
CN113918832B (en) Graph convolution collaborative filtering recommendation system based on social relationship
CN113918833B (en) Product recommendation method realized through graph convolution collaborative filtering of social network relationship
CN113918834B (en) Graph convolution collaborative filtering recommendation method fusing social relations
Guo et al. Attention based consistent semantic learning for micro-video scene recognition
Chen et al. CoNet: Co-occurrence neural networks for recommendation
CN116894476A (en) Multi-behavior attention self-supervision learning method based on double channels
CN113590976A (en) Recommendation method of space self-adaptive graph convolution network
Hao et al. Multi-dimensional graph neural network for sequential recommendation
CN116992151A (en) Online course recommendation method based on double-tower graph convolution neural network
Khan et al. Cross‐modal retrieval based on deep regularized hashing constraints
Wang et al. DSDRec: Next POI recommendation using deep semantic extraction and diffusion model
CN106096653B (en) Ascribed characteristics of population estimating method based on cross-platform user social contact multimedia behavior
CN111368552A (en) Network user group division method and device for specific field
US20240184835A1 (en) Recommender system and its operation
CN115795005A (en) Session recommendation method and device integrating contrast learning denoising optimization
Yang et al. Graph clustering with embedding propagation
CN117521770A (en) Training method and device for node detection model and computer equipment
CN113010772A (en) Data processing method, related equipment and computer readable storage medium
Doan et al. Pacela: A neural framework for user visitation in location-based social networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination