CN116894476A - Multi-behavior attention self-supervision learning method based on double channels - Google Patents

Abstract

The present invention proposes a multi-behavior attention self-supervised learning method based on dual channels. The method is able to distinguish the ability of different user behavior perception preferences. Take advantage of the different behaviors of users interacting with items. Capturing the user's dependence on multiple behaviors, the three self-supervised learning methods invented not only enhance the representation results of the dual channels, but also enable the model to obtain more auxiliary supervision signals in self-supervised learning within and between channels. Effectively alleviates the problem of sparse supervision signals.

Description

A self-supervised learning method for multi-behavior attention based on dual channels

Technical field

The invention relates to the technical field of multiple behavioral preference predictions based on dual-channel cross-behavior dependency modeling, and specifically relates to a bidirectional coding representation converter and a graph neural network and a personalized behavior recommendation method of self-supervised learning. In particular, it relates to a multi-behavior attention self-supervised learning method based on dual channels.

Background technique

In recent years, with the development of the mobile Internet, the focus of e-commerce has shifted from personal computers to smartphones. Various mobile e-commerce platforms have emerged. Training networks based on user interaction have also made great progress, especially multi-behavior recommendation. attracted widespread attention from academic circles. The massive interaction data generated by millions of users provides an excellent opportunity to explore the potential intentions of users in various behaviors. However, at the same time, the massive data also puts users into the dilemma of sparse behavioral information types, making it impossible to effectively make personalized recommendations. In order to solve this problem, multi-behavior recommendation has become one of the most popular research topics in the field of e-commerce platform mining as an effective method that not only helps users explore the products they are interested in but also helps e-commerce platforms attract more potential users. one.

Generally, since typing on mobile devices is more difficult than on desktop computers, intent recommendations can save users' time without requiring any input, which will increase user activity and shopping experience, and users can communicate with friends through sharing behaviors Sharing their favorite things, these behaviors contain user contextual information and often reflect user interests. Through in-depth mining of this behavioral information, deeper user preferences for products can be revealed. The user's interactive behavior represents the interaction between the user and the product, and also shows the product characteristics and user characteristics. From the user's perspective, the user clicks on the product and adds it to the shopping cart, and finally liquidates multiple products in the shopping cart together; from the product's perspective, for products with interactive behaviors including additional purchases and collections, only a single Products with interactive behaviors are more attractive to this user. In addition, users often share products with friends. These unique behaviors make multi-behavior recommendation different from traditional recommendation systems. Therefore, it is necessary to comprehensively understand the interactive information of users in multiple behaviors for products, so as to develop new algorithms for multi-behavior recommendation.

Contents of the invention

The purpose of the present invention is to solve the problems in the existing technology and thereby propose a multi-behavior attention self-supervised learning method based on dual channels.

The present invention is realized through the following technical solutions. The present invention proposes a multi-behavior attention self-supervised learning method based on dual channels. The method includes the following steps:

Step 1. Obtain the product interaction data set from Tmall and CIKM2019 E-commerce Artificial Intelligence Challenge, select T% of the data as the training data set, and (1-T%) of the data as the test data set. The training data set contains user interactions. Products, users, and users’ various interactive behavior histories;

Step 2. The user set in the training data set is U, U={u ₁ , u ₂ ,..., u _q , _. .., u _N }, q∈{1,..., N}, where u _q is the qth user, N is the number of users; the product set is I, I={i ₁ , i ₂ ,..., it _t ,..., i _T }, t∈{1,... , T}, where i _t is the t-th user, T is the quantity of goods; the behavior set is B, B={b ₁ , b ₂ ,..., b _k,. ..., i _K }, k ∈{1,...,K}, where b _k is the k-th behavior and K is the number of behaviors;

Step 3: From the perspective of sequence channels, construct a user-product interaction sequence based on the user's behavior history;

Step 4. From the perspective of sequence channels, since the deep bidirectional model is better than the unidirectional model, the calculation method of BERT4Rec is introduced. GELU is the Gaussian error linear unit activation function; W represents the weight matrix of the GELU activation function, and b represents the bias;/> Softmax is used as the output activation function to normalize the results of splicing various behavioral sequences. For different users, they have different behavioral interaction sequences that lead to different encoding results;

Step 5: Considering the sequence channel and based on the above characterization results, design the self-supervised loss;

Step 6: Consider the graph channel to obtain enough available information about the user; users have multiple behaviors, including clicking, adding to shopping cart, collecting and purchasing behaviors; define G = (V, E), V means that the node set contains the user The set u∈U and the item set i∈I are (U, I)∈V; E represents the different interactive behaviors between user nodes and item nodes; the embedding of multi-behavior graphs consists of multiple behavior sub-graph embeddings, and the behavior sub-graph Graph embedding is expressed as G _b =(V _b , E _b );

Step 7: Consider from the diagram channel, auxiliary behavior diagram and target behavior diagram/> as input to attention;

Step 8: Considering the graph channel, design self-supervised learning of multi-behavior interaction graphs in the channel, and enhance multiple behavioral data supervision signals through self-supervised learning;

Step 9: Consider the sequence channel and the graph channel, and enhance the supervision signal through self-supervised learning combined with dual channels.

Further, in step three, each element in the user interaction behavior sequence is set as a triplet feature vector. Indicates that user q uses the kth behavior to interact with item It contains the products that user q interacts with through all behaviors; the feature vector of the auxiliary behavior interaction sequence/> Feature vector of interaction sequence with target behavior/> As input to a multi-behavior interaction sequence-dependent encoder.

Further, calculate the feature vector of each auxiliary behavior and the feature vector of the target behavior. The calculation process is: Where W ^Q , W ^Q ∈R ^d*n is the weight matrix of the learnable behavior vector;/> Express/> transposition;/> Represents the correlation matrix between auxiliary behavior k and target behavior k′;/> Each correlation matrix/> After softmax normalization, the attention score that conforms to the value range of the probability distribution is obtained/> softmax calculates the cosine similarity to find the behavior closest to the purchase behavior; W ^V ∈R ^d*n is the weight matrix of learnable behavior vectors.

Further, in step five, different behaviors of the same user are regarded as positive sample pairs Different behaviors between different users are treated as negative sample pairs/> Therefore, the self-monitoring loss regarding user behavior is:/> Among them/> middle Represents the calculation of cosine similarity.

Further, in step six, graph convolution is used to learn the node representation of the graph, aggregate and transfer node features; the process of graph convolution is specifically: for each behavioral subgraph, it is embedded into an adjacency matrix A _k , which is It is composed of matrix R _k , and the specific process is: Each behavioral subgraph is embedded into an adjacency matrix A _k as the input of the normalized Laplacian matrix of the behavior. The normalization process is:/> Among them/> The degree matrix characterizing k behavior, I _k represents the identity matrix of k behavior/> The output of graph convolution passes through the threshold function sigmoid:/> Among them/> is the node feature matrix of the l layer of the node in the graph, W _k is the transformation matrix for behavioral view information transmission; graph convolution has a total of L layers, L represents the obtained L-order neighbor nodes, and information aggregation is obtained through node information The process of obtaining the characteristics of nodes related to k kinds of behaviors in the graph can save multi-behavior context information.

Further, in step seven, the process of identifying the intensity of the influence of the auxiliary behavior map on the target behavior map through attention is: Where W ^Q ∈R ^d*n and W ^K ∈R ^d*n are the weight matrices of the behavior matrix that can be continuously updated iteratively, /> Is the attention correlation coefficient matrix;/> The attention calculation process and/> The attention calculation process is the same and is regarded as the weight multiplied by the auxiliary behavior/> Where W ^V ∈R ^d*n is the weight matrix of the behavior matrix that can be continuously updated iteratively,/> is the auxiliary behavior feature matrix for the target behavior, which is the final output of the cross-behavior interaction graph attention encoder.

Further, in step eight, different behavior views of the same user are regarded as positive sample pairs Different behaviors of different users are regarded as negative sample pairs/> Maximize the mutual information between users by self-supervising pairs of positive and negative samples: The consistency of the two behavioral views and maximizing the differences between different user behaviors are enhanced by behavioral data supervision signals.

Further, in step nine, the sequence channel and view channel of the same user are regarded as positive samples, using Represented; the sequence channels and view channels of different users are regarded as negative samples, using/> Represents; self-monitoring loss:/> τ is the temperature coefficient, balancing the intensity of learning between the two channels; the sum of all self-supervised losses is used as the final target loss: L _CL = L _SCL + L _GCL + L _SGCL ; L _SCL is the multi-behavior interactive sequence self-supervised loss in the sequence channel ; L _GCL is the self-supervised loss of the multi-behavior interaction graph within the graph channel; the final loss function list L _CL consists of the sequence loss function L _ScL and the view loss function L _GCL and the sequence view loss function L _SGCL for each pair of behaviors.

The present invention proposes an electronic device, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the dual-channel-based multi-behavior attention self-supervised learning method. step.

The present invention proposes a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the dual-channel-based multi-behavior attention self-supervised learning method are implemented.

The invention has the following beneficial effects:

The present invention proposes a dual-channel multi-behavior attention self-supervised learning method, which can distinguish the capabilities of different users' behavior perception preferences. Take advantage of the different behaviors of users interacting with items in different modes. Capturing the commonality of users' multiple behaviors, the three self-supervised learning methods invented not only enhance the representation results of dual channels, but also enable the model to obtain more auxiliary supervision signals in self-supervised learning within and between channels, effectively Alleviating the problem of sparse supervision signals.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts.

Figure 1 is an overall schematic diagram of a multi-behavior attention self-supervised learning method based on dual channels of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Personalized recommendations are an important part of e-commerce platforms. The recommendation system enhances collaborative filtering through neural networks to accurately capture user preferences, thereby obtaining better recommendation performance. Traditional recommendation methods focus on the results of a single user behavior and ignore the modeling of users' multiple interactive behaviors (click, add to shopping cart, purchase). Although many studies have also focused on multi-behavior modeling, there are still two important challenges: 1) Due to the neglect of multiple behavioral context information, there are still challenges in identifying multi-modal relationships of behaviors. 2) Supervision signals are still sparse. In order to solve this problem, the present invention proposes a multi-behavior attention dual-channel contrastive learning method. Through set self-supervised learning, multiple behavioral context information is extracted from different types of interactions of users, providing behavioral dependencies for users to obtain different behaviors. of multiple relationships. Enhance the robustness of the model. The present invention designs dual-channel self-supervised learning to enhance the data supervision signal. Extensive experiments on two real-world datasets show that our method consistently outperforms state-of-the-art multi-behavior recommendation methods.

Combined with Figure 1, the present invention proposes a multi-behavior attention self-supervised learning method based on dual channels. The method includes the following steps:

Step 1. Obtain the product interaction data set from Tmall and CIKM2019 E-commerce Artificial Intelligence Challenge, select T% of the data as the training data set, and (1-T%) of the data as the test data set. The training data set contains user interactions. Products, users, and users’ various interactive behavior histories;

Step 2. The user set in the training data set is U, U={u ₁ , u ₂ ,..., u _q , _. .., u _N }, q∈{1,..., N}, where u _q is the qth user, N is the number of users; the product set is I, I={i ₁ , i ₂ ,..., it _t ,..., i _T }, t∈{1,... ., T}, where i _t is the t-th user, T is the quantity of goods; the behavior set is B, B={b ₁ , b ₂ ,..., b _k ,...., i _K }, k∈{1,...,K}, where b _k is the k-th behavior and K is the number of behaviors;

Step 3: From the perspective of sequence channels, construct a user-product interaction sequence based on the user's behavior history;

Step 4. From the perspective of sequence channels, since the deep bidirectional model is better than the unidirectional model, the calculation method of BERT4Rec is introduced. GELU is the Gaussian error linear unit activation function; W represents the weight matrix of the GELU activation function, and b represents the bias;/> Softmax is used as the output activation function to normalize the results of splicing various behavioral sequences. For different users, they have different behavioral interaction sequences that lead to different encoding results;

Step 5: Considering the sequence channel and based on the above characterization results, design the self-supervised loss;

Step 6: Consider the graph channel to obtain enough available information about the user; users have multiple behaviors, including clicking, adding to shopping cart, collecting and purchasing behaviors; define G = (V, E), V means that the node set contains the user The set u∈U and the item set i∈I are (U, I)∈V; E represents the different interactive behaviors between user nodes and item nodes; the embedding of multi-behavior graphs consists of multiple behavior sub-graph embeddings, and the behavior sub-graph Graph embedding is expressed as G _b =(V _b , E _b );

Step 7: Consider from the diagram channel, auxiliary behavior diagram and target behavior diagram/> as input to attention;

Step 8: Considering the graph channel, design self-supervised learning of multi-behavior interaction graphs in the channel, and enhance multiple behavioral data supervision signals through self-supervised learning;

Step 9: Consider the sequence channel and the graph channel, and enhance the supervision signal through self-supervised learning combined with dual channels.

In step three, set each element in the user interaction behavior sequence to a triplet feature vector. Indicates that user q uses the k-th behavior to interact with item It contains the products that user q interacts with through all behaviors; the feature vector of the auxiliary behavior interaction sequence/> Feature vector of interaction sequence with target behavior/> As input to a multi-behavior interaction sequence-dependent encoder.

Calculate the feature vector of each auxiliary behavior and the feature vector of the target behavior. The calculation process is: Where WQ, WQ∈Rd*n is the weight matrix of the learnable behavior vector;/> express transposition;/> Represents the correlation matrix between auxiliary behavior k and target behavior k′;/> Each correlation matrix/> After softmax normalization, the attention score that conforms to the value range of the probability distribution is obtained/> softmax calculates the cosine similarity to find the behavior closest to the purchase behavior; WV∈Rd*n is the weight matrix of learnable behavior vectors.

In step five, different behaviors of the same user are regarded as positive sample pairs Different behaviors between different users are treated as negative sample pairs/> Therefore, the self-monitoring loss regarding user behavior is: Among them/> in/> Represents the calculation of cosine similarity.

In step six, graph convolution is used to learn the node representation of the graph, aggregate and transfer node features; the process of graph convolution is specifically: for each behavioral subgraph, it is embedded into an adjacency matrix A _k , which is represented by the matrix R _k is formed, the specific process is: Each behavioral subgraph is embedded into an adjacency matrix A _k as the input of the normalized Laplacian matrix of the behavior. The normalization process is:/> Among them/> The degree matrix characterizing k behavior, I _k represents the identity matrix of k behavior The output of graph convolution passes through the threshold function sigmoid:/> Among them/> is the node feature matrix of the l layer of the node in the graph, W _k is the transformation matrix for behavioral view information transmission; graph convolution has a total of L layers, L represents the obtained L-order neighbor nodes, and information aggregation is obtained through node information The process of obtaining the characteristics of nodes related to k kinds of behaviors in the graph can save multi-behavior context information.

In step seven, the process of identifying the intensity of the influence of the auxiliary behavior map on the target behavior map through attention is: Where W ^Q ∈R ^d*n and W ^K ∈R ^d*n are the weight matrices of the behavior matrix that can be continuously updated iteratively, /> Is the attention correlation coefficient matrix;/> The attention calculation process of The attention calculation process is the same and is regarded as the weight multiplied by the auxiliary behavior/> Where W ^V ∈R ^d*n is the weight matrix of the behavior matrix that can be continuously updated iteratively,/> is the auxiliary behavior feature matrix for the target behavior, which is the final output of the cross-behavior interaction graph attention encoder.

In step eight, different behavioral views of the same user are regarded as positive sample pairs Different behaviors of different users are regarded as negative sample pairs/> Maximize mutual information between users by self-supervising pairs of positive and negative samples: /> The consistency of the two behavioral views and maximizing the differences between different user behaviors are enhanced by behavioral data supervision signals.

In step nine, the sequence channel and view channel of the same user are regarded as positive samples, using Represented; the sequence channels and view channels of different users are regarded as negative samples, using/> Represents; self-monitoring loss:/> τ is the temperature coefficient, balancing the intensity of learning between the two channels; the sum of all self-supervised losses is used as the final target loss: L _CL = L _SCL + L _GCL + L _SGCL ; L _SCL is the multi-behavior interactive sequence self-supervised loss in the sequence channel ; L _GCL is the self-supervised loss of the multi-behavior interaction graph within the graph channel; the final loss function list L _CL consists of the sequence loss function L _SCL and the view loss function L _GCL and the sequence view loss function L _SGCL for each pair of behaviors.

Example

The present invention proposes a general and flexible multi-behavior relationship learning framework—a dual-channel multi-behavior attention self-supervised learning method. Specifically, the described method first proposes a multi-action dependency encoder to learn the interdependencies of actions by combining specific types of action representations in different types of user-item interactions. Then dual-channel multi-behavior self-supervised learning solves the data sparse problem. To model multi-type behavior pattern dependencies and conduct comprehensive learning for recommendation. The dual-channel multi-behavior dependent self-supervised learning model designed by the present invention is to parameterize each type of user-item interaction into a separate embedding space to learn the dependency representation of the user's personalized behavior type, and utilize the inter-channel self-supervised learning paradigm to enhance Data surveillance signals.

The present invention proposes a multi-behavior attention self-supervised learning method based on dual channels. The method includes the following steps:

Step 1. Obtain the product interaction data set from Tmall and CIKM2019 E-commerce Artificial Intelligence Challenge. Select T% of the data as training data and (1-T%) of the data as test data. The training data set includes users’ responses to products, users and the user’s various interactive behavior histories;

Step 2. The user set in the training set is U, U={u ₁ , u ₂ ,..., u _q ,..., u. _N }, q∈{1,..., N}, where u _q is the q-th user, and N is the number of users. The product set is I, I={i ₁ , i ₂ ,..., it _t ,..., i _T }, t∈{1,..., T}, where i _t is the t-th user , T is the quantity of goods. The behavior set is B, B={b ₁ , b ₂ , ..., b _k , _. ..., i _K }, k∈{1, ..., K}, where b _k is the k-th behavior , K is the number of behaviors.

Step 3. From the perspective of sequence channels, construct a user-product interaction sequence based on the user's behavior history. Set each element in the user interaction behavior sequence as a triplet feature vector Indicates that user q uses the kth behavior to interact with item x. A user's multi-behavior sequence contains interaction information of a single user. As a result, the user's multi-behavior interaction sequence is mapped into the initial embedding to form a feature matrix/> It contains the goods that user q interacts with through all actions. Characteristic vector of auxiliary behavior interaction sequence/> Feature vector of interaction sequence with target behavior/> As the input of the multi-behavior interaction sequence dependence encoder, the calculation method of the multi-behavior interaction sequence dependence encoder is consistent with the calculation method of attention. The feature vector of each auxiliary behavior and the feature vector of the target behavior are calculated. The calculation process is/> where W ^Q , W ^Q ∈R ^d*n is the weight matrix of learnable behavior vectors. /> Express/> of transposition. /> Represents the correlation matrix between auxiliary behavior k and target behavior k′. /> Each correlation matrix/> After softmax normalization, the attention score that conforms to the value range of the probability distribution is obtained/> Softmax calculates the cosine similarity to find the behavior closest to the purchase behavior. W ^V ∈R ^d*n is the weight matrix of learnable behavior vectors. In order to prevent over-fitting problems and avoid excessive computational time costs, we use dropout to get/>

Step 4. From the perspective of sequence channels, since the deep bidirectional model is better than the unidirectional model, the calculation method of BERT4Rec is introduced. GELU is the Gaussian error linear unit activation function. W represents the weight matrix of the GELU activation function, and b represents the bias. /> Softmax is used as the output activation function to normalize the results of splicing various behavioral sequences. For different users, they have different behavioral interaction sequences resulting in different coding results.

Step 5. Considering the sequence channel, a self-supervised loss is designed for the above characterization results. Specifically, different behaviors of the same user are regarded as positive sample pairs Different behaviors between different users are treated as negative sample pairs Therefore, the self-monitoring loss regarding user behavior is:/> Among them/> in/> Represents the calculation of cosine similarity.

Step 6. Considering the graph channel, it is impossible to obtain enough available information using the user's single information. Because users have a variety of behaviors, including clicks, adding to shopping carts, collections and purchasing behaviors. Define G = (V, E), V means that the node set contains the user set u∈U and the item set i∈I, that is, (U, I)∈V. E represents the different interactive behaviors between user nodes and project nodes. In addition, the embedding of a multi-behavior graph is composed of multiple behavior sub-graph embeddings, so the behavior sub-graph embedding can be expressed as G _b = (V _b , E _b ). For example, the behavior subgraph composed of the items clicked by the user, G _click = (V _click , E _click ), where G _click represents the item view representation of the user's interaction through the click behavior, and V _click represents the user and item nodes connected to the click behavior. E _click represents the user's click behavior. First, graph convolution is dedicated to learning the node representation of the graph, aggregating and transmitting node features. The process of graph convolution, specifically, embeds each behavioral subgraph into an adjacency matrix A _k , which is composed of the matrix R _k . The specific process is as follows: Each behavioral subgraph is embedded into an adjacency matrix A _k as the input of the normalized Laplacian matrix of the behavior. The normalization process is as follows: /> Among them/> The degree matrix characterizing k behavior, I _k represents the identity matrix of k behavior Since graph convolution can well obtain the high-order dependencies of user nodes, graph convolution can be used to better obtain the global representation of all user nodes for the user's multi-behavior interaction graph. The output method of graph convolution passes through the threshold function/> Among them/> is the node feature matrix of layer l of the node in the graph, and W _k is the transformation matrix for behavioral view information transmission. Graph convolution has a total of L layers, where L represents the acquired L-order neighbor nodes. The process of information aggregation is obtained through node information, and the characteristics of nodes related to k behaviors in the graph are obtained, and multi-behavior context information can be saved.

Step 7. Consider from the diagram channel, auxiliary behavior diagram and target behavior diagram/> as attentional input. The process of identifying the intensity of the influence of the auxiliary behavior map on the target behavior map through attention is as follows: /> Where W ^Q ∈R ^d*n and W ^K ∈R ^d*n are the weight matrices of the behavior matrix that can be continuously updated iteratively, /> is the attention correlation coefficient matrix. The attention calculation process is the same as the attention calculation process in step 3, which is regarded as the weight multiplied by the auxiliary behavior/> Where W ^V ∈R ^d*n is the weight matrix of the behavior matrix that can be continuously updated iteratively. /> is the auxiliary behavior feature matrix for the target behavior, which is the final output of the cross-behavior interaction graph attention encoder.

Step 8. Considering the graph channel, self-supervised learning of multi-behavior interaction graphs within the channel is designed, and multiple behavioral data supervision signals are enhanced through self-supervised learning. Different behavioral views of the same user are considered positive sample pairs Different behaviors of different users are regarded as negative sample pairs/> Maximize the mutual information between users by self-supervising pairs of positive and negative samples: Consistency of two behavioral views. And maximize the difference between different user behaviors, and obtain the enhancement of behavioral data supervision signals.

Step 9. Considering the sequence channel and the graph channel, self-supervised learning combined with dual channels is more conducive to enhancing the supervision signal. Consider the sequence channel and view channel of the same user as positive samples, use express. Sequence channels and view channels of different users are regarded as negative samples, use/> express. Self-supervised loss: τ is the temperature coefficient, balancing the intensity of learning between the two channels. The sum of all self-supervised losses is the final target loss: L _CL = L _SCL + L _GCL + L _SaCL . L _SCL is the multi-behavior interactive sequence self-supervised loss within the sequence channel mentioned in the step 5 section. L _GCL is a self-supervised loss for multi-behavior interaction graphs within the graph channel mentioned in the step 9 section. Therefore, the present invention supplements the two proposed self-supervision losses. The final loss function list L _CL consists of the sequence loss function L _SCL and the view loss function L _GCL and the sequence view loss function L _SGCL for each pair of behaviors.

The present invention proposes an electronic device, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the dual-channel-based multi-behavior attention self-supervised learning method. step.

The present invention proposes a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the dual-channel-based multi-behavior attention self-supervised learning method are implemented.

The memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be read only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (erasablePROM, EPROM), electrically erasable memory Programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of illustration, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM) ), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) and Direct memory bus random access memory (direct rambusRAM, DR RAM). It should be noted that the memory of the method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., the computer instructions may be transferred from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The usable media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVD)), or semiconductor media (e.g., solid state disks). SSD)) etc.

During the implementation process, each step of the above method can be completed by instructions in the form of hardware integrated logic circuits or software in the processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

It should be noted that the processor in the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the above method embodiment can be completed through an integrated logic circuit of hardware in the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. . Each method, step and logical block diagram disclosed in the embodiment of this application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

The above is a detailed introduction to the multi-behavior attention self-supervised learning method based on dual channels proposed by the present invention. In this article, specific examples are used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for To help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the field, there will be changes in the specific implementation and application scope based on the idea of the present invention. In summary, this specification The contents should not be construed as limitations of the invention.

Claims (10)

1. A dual-channel based multi-behavior attention self-supervised learning method, characterized in that: the method includes the following steps: Step 1. Obtain the product interaction data set from Tmall and CIKM2019 E-commerce Artificial Intelligence Challenge, select T% of the data as the training data set, and (1-T%) of the data as the test data set. The training data set contains user interactions. Products, users, and users’ various interactive behavior histories; Step 2. The user set in the training data set is U, U={u ₁ ,u ₂ ,...,u _q ,...,u _N }, q∈{1,...,N}, where u _q is the qth user, N is the number of users; the product set is I, I={i ₁ ,i ₂ ,...,i _t ,....,i _T },t∈{1,.. .,T}, where i _t is the t-th user, T is the quantity of goods; the behavior set is B, B={b ₁ ,b ₂ ,...,b _k ,....,i _K }, k∈{1,...,K}, where b _k is the k-th behavior and K is the number of behaviors; Step 3: From the perspective of sequence channels, construct a user-product interaction sequence based on the user's behavior history; Step 4. From the perspective of sequence channels, since the deep bidirectional model is better than the unidirectional model, the calculation method of BERT4Rec is introduced. GELU is the Gaussian error linear unit activation function; W represents the weight matrix of the GELU activation function, and b represents the bias;/> Softmax is used as the output activation function to normalize the results of splicing various behavioral sequences. For different users, they have different behavioral interaction sequences that lead to different encoding results; Step 5: Considering the sequence channel and based on the above characterization results, design the self-supervised loss; Step 6: Consider the graph channel to obtain enough available information about the user; users have multiple behaviors, including clicking, adding to shopping cart, collecting and purchasing behaviors; define G = (V, E), V means that the node set contains the user The set u∈U and the item set i∈I are (U,I)∈V; E represents the different interactive behaviors between user nodes and item nodes; the embedding of multi-behavior graphs consists of multiple behavior sub-graph embeddings, and the behavior sub-graph embedding Graph embedding is expressed as G _b =(V _b ,E _b ); Step 7: Consider from the diagram channel, auxiliary behavior diagram and target behavior diagram/> as input to attention; Step 8: Considering the graph channel, design self-supervised learning of multi-behavior interaction graphs in the channel, and enhance multiple behavioral data supervision signals through self-supervised learning; Step 9: Consider the sequence channel and the graph channel, and enhance the supervision signal through self-supervised learning combined with dual channels. 2. The method according to claim 1, characterized in that: in step three, each element in the user interaction behavior sequence is set to be a triplet feature vector. Indicates that user q uses the kth behavior to interact with item It contains the products that user q interacts with through all behaviors; the feature vector of the auxiliary behavior interaction sequence/> Feature vector of interaction sequence with target behavior/> As input to a multi-behavior interaction sequence-dependent encoder. 3. The method according to claim 2, characterized in that: calculating the feature vector of each auxiliary behavior and the feature vector of the target behavior, the calculation process is: Where W ^Q , W ^Q ∈R ^d*n is the weight matrix of the learnable behavior vector;/> Express/> transposition;/> Represents the correlation matrix between auxiliary behavior k and target behavior k';/> Each correlation matrix/> After softmax normalization, the attention score that conforms to the value range of the probability distribution is obtained/> softmax calculates the cosine similarity to find the behavior closest to the purchase behavior;/> W ^V ∈R ^d*n is the weight matrix of learnable behavior vectors. 4. The method according to claim 3, characterized in that: in step five, different behaviors of the same user are regarded as positive sample pairs. Different behaviors between different users are treated as negative sample pairs/> U,q≠p}; thus the self-supervision loss regarding user behavior is:/> Among them/> in/> Represents the calculation of cosine similarity. 5. The method according to claim 4, characterized in that: in step six, graph convolution is used to learn the node representation of the graph, aggregate and transfer node features; the process of graph convolution is specifically: for each The behavioral subgraph is embedded into the adjacency matrix A _k , which is composed of the matrix R _k . The specific process is: Each behavioral subgraph is embedded into an adjacency matrix A _k as the input of the normalized Laplacian matrix of the behavior. The normalization process is:/> Among them/> The degree matrix characterizing k behavior, I _k represents the identity matrix of k behavior/> The output of graph convolution passes through the threshold function Among them/> is the node feature matrix of the l layer of the node in the graph, W _k is the transformation matrix for behavioral view information transmission; graph convolution has a total of L layers, L represents the obtained L-order neighbor nodes, and information aggregation is obtained through node information The process of obtaining the characteristics of nodes related to k kinds of behaviors in the graph can save multi-behavior context information. 6. The method according to claim 5, characterized in that: in step seven, the process of identifying the influence intensity of the auxiliary behavior map on the target behavior map through attention is: Where W ^Q ∈R ^d*n and W ^K ∈R ^d*n are the weight matrices of the behavior matrix that can be continuously updated iteratively, /> Is the attention correlation coefficient matrix;/> The attention calculation process and/> The attention calculation process is the same and is regarded as the weight multiplied by the auxiliary behavior/> Where W ^V ∈R ^d*n is the weight matrix of the behavior matrix that can be continuously updated iteratively,/> is the auxiliary behavior feature matrix for the target behavior, which is the final output of the cross-behavior interaction graph attention encoder. 7. The method according to claim 6, characterized in that: in step eight, different behavior views of the same user are regarded as positive sample pairs Different behaviors of different users are regarded as negative sample pairs/> Maximize mutual information between users by self-supervising pairs of positive and negative samples: /> The consistency of the two behavioral views and maximizing the differences between different user behaviors are enhanced by behavioral data supervision signals. 8. The method according to claim 7, characterized in that: in step nine, the sequence channel and the view channel of the same user are regarded as positive samples, using Represented; the sequence channels and view channels of different users are regarded as negative samples, using/> Represents; self-monitoring loss:/> τ is the temperature coefficient, balancing the intensity of learning between the two channels; the sum of all self-supervised losses is used as the final target loss: L _CL = L _SCL + L _GCL + L _SGCL ; L _SCL is the multi-behavior interactive sequence self-supervised loss in the sequence channel ; L _GCL is the self-supervised loss of the multi-behavior interaction graph within the graph channel; the final loss function list L _CL consists of the sequence loss function L _SCL and the view loss function L _GCL and the sequence view loss function L _SGCL for each pair of behaviors. 9. An electronic device, including a memory and a processor, the memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the method of any one of claims 1-8 are implemented. 10. A computer-readable storage medium used to store computer instructions, characterized in that when the computer instructions are executed by a processor, the steps of the method described in any one of claims 1-8 are implemented.