CN115730143A - Recommendation system, method, terminal and medium based on task alignment meta learning and augmentation graph - Google Patents

Abstract

The invention provides a recommendation system, method, terminal and medium based on task alignment meta-learning and augmented graph, and provides a TMAG (time map gateway) to solve the cold start problem of a model level and a feature level at the same time. On the model level, a task alignment constructor is provided to capture potential clustering knowledge which can be quickly adapted to a new user, so that the local optimal problem is solved; task-level attributes are also employed to enhance the underlying clustering knowledge versus regularization terms. At the aspect of characteristics, the adjacency matrix of the graph is expanded by combining graph structure information and attribute information, so that the data sparsity problem is relieved. Extensive experiments on three real world datasets demonstrate the effectiveness of the model of the present invention in cold start recommendations.

Description

Recommendation system, method, terminal and medium based on task alignment meta learning and augmentation graph

Technical Field

The present application relates to the field of cold start and recommendation technologies, and in particular, to a recommendation system, method, terminal, and medium based on task aligned meta learning and augmented graph.

Background

Recommendation systems are intended to discover the interests of users and have been widely applied to various online systems, such as e-commerce platforms, online advertising, and social platforms. Although the traditional Matrix Factorization (MF) model and the popular deep learning model have succeeded, one of the major challenges facing most recommendation systems is the cold start problem due to the lack of user-item interaction. Solving this problem is important because new users may give up the system when they initially receive a poor recommendation.

The traditional approach to alleviating the cold start problem is to use a feature level strategy, which can be divided into two broad categories. The first is to model intrinsic information (e.g., user representation, item properties, and cross-domain knowledge) to enhance the representation of a new user or new item; the second category is modeling feature interactions through Graph Neural Networks (GNNs) and Heterogeneous Information Networks (HINs) to capture higher-order cooperative signals. Despite the advances, these approaches address the cold start problem from the feature level, which depends largely on the availability and quality of the features.

On the other hand, at the model level, recently, work based on few-sample learning and meta-learning has made significant progress in solving the data sparseness problem in various fields. Most current meta-learning approaches employ optimization-based algorithms (e.g., MAML) to solve the cold start problem. The main idea of this method is to learn a global parameter to initialize the parameters of the personalized model. The method constructs various small sample user preference tasks simulating cold start scenes, and extracts meta-knowledge in the meta-training task as strong generalization prior. The learned a priori knowledge can then quickly adapt to new users with rare interactions during the meta-test.

However, the existing meta learning method has the following limitations. They see each user as a task and learn globally shared meta-knowledge among all users. When dealing with users with gradient descent methods different from the primary user, the coarse-grained global knowledge traps the model in local optimality. As shown in fig. 1A, a conventional meta-learning framework is shown, due to age differences,

the direction of gradient descent is dominant and therefore the parameter θ is biased towards a sub-optimal solution. Furthermore, existing methods do not take full advantage of inherent information and feature interactions, which are critical to the modeling of new users and items.

Disclosure of Invention

In view of the above-mentioned shortcomings of the prior art, the present application aims to provide a recommendation system, method, terminal and medium based on task aligned meta learning and augmented graph, so as to solve the technical problem that the prior art does not fully utilize the advantages of inherent information and feature interaction to make recommendations.

To achieve the above and other related objects, a first aspect of the present application provides a recommendation system based on task aligned meta learning and augmented graph, comprising: the task alignment constructor module is used for taking the user portrait and the article content as input parameters and outputting corresponding user characteristic embedding and article characteristic embedding; dividing users into different user clusters based on user characteristic embedding and article characteristic embedding, and generating a support set and a query set according to the user clusters to form a task cluster; the augmented graph neural network module is used for constructing a user-article bipartite graph based on the task cluster and capturing high-order user-article interaction information; generating potential interaction for the user based on a preset strategy; embedding the final layer diagram of the user or the article into corresponding attribute embedding and splicing to obtain the final embedding of the user or the article; and the contrast regularization module is used for enhancing the potential clustering knowledge by adopting the task attributes to contrast the regularization items.

In some embodiments of the first aspect of the present application, the task alignment constructor module comprises a self-encoding sub-module; the self-encoding submodule comprises an attribute-oriented self-encoder; the objective function of the self-encoder comprises:

wherein, W _u Representing all trainable model parameters; lambda control L ₂ The strength is normalized to prevent overfitting.

In some embodiments of the first aspect of the present application, the task alignment constructor module comprises a task construction sub-module; the task construction sub-module divides the users learned from the self-coding sub-module into different user clusters by using a K-Means algorithm, and generates a support set and a query set based on the user clusters to form a task cluster.

In some embodiments of the first aspect of the present application, the augmented graph neural network module includes a graph embedding propagation sub-module, a graph augmentation generator sub-module, a model prediction sub-module.

In some embodiments of the first aspect of the present application, the graph embedding propagation submodule constructs a user-item bipartite graph after obtaining the task clusters from the task construction submodule, and executes the GCN to capture high-order structure information of the interaction graph using interactions of the user in the task as training data.

In some embodiments of the first aspect of the present application, the graph augmentation generator sub-module is configured to generate potential interactions for the user based on any one of the following policies: the strategy is to mine the structure of the interaction graph and capture the potential dependency relationship of the user-item pair which does not appear on the way; and the second strategy is to represent the user by using the interactive item and add the potential item according to the similarity of the interactive item and the attribute of the user.

In some embodiments of the first aspect of the present application, the model prediction sub-module is configured to splice together the final L-layer graph embedding of the user u and its corresponding attribute embedding, so as to perform modeling in a finer-grained based manner.

In some embodiments of the first aspect of the present application, the recommendation system comprises setting a jointly optimized loss function as follows:

L＝L _pre +λ ₁ L _gen +λ ₂ L _MI +λ ₃ ||⊙|| ² ；

wherein, <' > indicates L _pre And L _gen All of the trainable parameters in (1), and L _MI No additional parameters were added; lambda [ alpha ] ₁ 、λ ₂ 、λ ₃ Parameterizing the weight of different losses; l is _pre Is to predict the lossA function; l is _gen Is a loss function for training the potential interaction generator; l is _MI Is a loss function of the regular contrast term.

In some embodiments of the first aspect of the present application, the updated support set and query set are obtained by performing a gradient descent update on the jointly optimized loss function.

To achieve the above and other related objects, a second aspect of the present application provides a recommendation method based on task aligned meta learning and augmented graph, including: taking the user portrait and the article content as input parameters, and outputting corresponding user characteristic embedding and article characteristic embedding; dividing users into different user clusters based on user characteristic embedding and article characteristic embedding, and generating a support set and a query set according to the user clusters to form a task cluster; constructing a user-article bipartite graph based on the task cluster and capturing high-level user-article interaction information; generating potential interaction for the user based on a preset strategy; embedding the final layer diagram of the user or the article into corresponding attribute embedding and splicing to obtain the final embedding of the user or the article; task attributes are used to enhance the potential clustering knowledge against the regularization terms.

To achieve the above and other related objects, a third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the recommendation method based on task aligned meta learning and augmented graph.

To achieve the above and other related objects, a fourth aspect of the present application provides an electronic terminal comprising: a processor and a memory; the memory is used for storing a computer program, and the processor is used for executing the computer program stored by the memory so as to enable the terminal to execute the recommendation method based on the task alignment meta learning and the augmentation graph.

As described above, the recommendation system, method, terminal and medium based on task aligned meta learning and augmented graph according to the present application have the following beneficial effects: the present invention proposes TMAG to solve the cold start problem at both the model level and the feature level. On the model level, a task alignment constructor is provided to capture potential clustering knowledge which can be quickly adapted to a new user, so that the local optimal problem is solved; task-level attributes are also employed to enhance the underlying clustering knowledge versus regularization terms. At the aspect of characteristics, the adjacency matrix of the graph is expanded by combining graph structure information and attribute information, so that the data sparsity problem is relieved. Extensive experiments on three real world datasets demonstrate the effectiveness of the model of the present invention in cold start recommendations.

Drawings

Fig. 1A is a schematic diagram of a conventional meta-learning framework.

FIG. 1B is a schematic diagram of a TMAG frame according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of a recommendation system based on task aligned meta learning and augmentation graph according to an embodiment of the present invention.

FIG. 3 is a diagram illustrating the effect of support set size in an embodiment of the present invention.

FIG. 4 is a diagram illustrating the effect of the amplification factor in an embodiment of the invention.

Fig. 5 is a flowchart illustrating a recommendation method based on task aligned meta learning and augmented graph according to an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of an electronic terminal according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.

It is noted that in the following description, reference is made to the accompanying drawings which illustrate several embodiments of the present application. It is to be understood that other embodiments may be utilized and that mechanical, structural, electrical, and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description is not to be taken in a limiting sense, and the scope of embodiments of the present application is defined only by the claims of the issued patent. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. Spatially relative terms, such as "upper," "lower," "left," "right," "lower," "below," "lower," "above," "upper," and the like, may be used herein to facilitate describing one element or feature's relationship to another element or feature as illustrated in the figures.

In this application, unless expressly stated or limited otherwise, the terms "mounted," "connected," "secured," "retained," and the like are to be construed broadly and encompass, for example, both fixed and removable connections or integral connections; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.

Also, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," and/or "comprising," when used in this specification, specify the presence of stated features, operations, elements, components, items, species, and/or groups, but do not preclude the presence, or addition of one or more other features, operations, elements, components, items, species, and/or groups thereof. The terms "or" and/or "as used herein are to be construed as inclusive or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a; b; c; a and B; a and C; b and C; A. b and C ". An exception to this definition will occur only when a combination of elements, functions or operations are inherently mutually exclusive in some way.

The invention provides a cold start recommendation method, a device, a terminal and a medium based on task alignment meta learning and an augmentation graph. In a first aspect, for the problem of how to alleviate the local optimality of meta-learning in a cold-start scenario, a fine-grained task alignment constructor provided in the present invention clusters similar users and partitions tasks for meta-learning. In particular, the attribute-oriented self-encoder extracts potential representations of users and items based on intrinsic attributes; users with similar representations are then clustered into a cluster, treated as a task and have consistent optimization directions, thus mitigating the local optimization problem in the meta-training process. In a second aspect, an augmented graph neural network is provided to capture high-order user-item interaction information for how to leverage intrinsic information and feature interactions to mitigate sparsity. Specifically, two graph augmentation methods are utilized to relieve data sparsity, and potential interaction signals are explored from the view of attributes and graph structures respectively.

The conventional model and the TMAG model proposed by the present invention are compared with fig. 1A and 1B: the traditional structure (fig. 1A) has a visualization of model parameters for 4 tasks, and the TMAG model has 2 tasks aligned by age. The gradient descent direction in the conventional meta-learning structure is biased to 4, whereas in the present invention, the same group of users have a consistent optimization direction, thereby avoiding the local optimization problem.

Before the present invention is explained in further detail, terms and expressions referred to in the embodiments of the present invention are explained, and the terms and expressions referred to in the embodiments of the present invention are applicable to the following explanations:

(1) Meta learning: a process known as learning, mines the underlying patterns behind user-item interactions, and aims to extract meta-knowledge across tasks, which can quickly adapt to new tasks through limited instances. Meta-learning learns general knowledge from a broad class of tasks and generalizes it to new tasks. In meta-learning, a task set T is defined, which trains the task T from the meta-training ^tr And meta test task T ^te And (4) forming. For each task T _k P (T) defines two sets, the support set S _k And query set Q _k . Meta learning based on optimizationThe method attempts to find the ideal parameters θ of the model f. During meta-training, T _k ∈T ^tr There are two rounds of updates. In inner loop update, the model passes

Gradient of (2)

Updating the parameter theta to theta _k Wherein L is _k (f _θ ) Representing the training loss for the task with the global parameter theta. Model training parameter minimization query set in outer loop update

Is/are as follows

Wherein

Is a task-specific parameter theta _k Task T of _k The test loss of (2). In the meta-test process, the model is only in the support set S ^te Updating parameters in inner loop and searching set Q ^te The above evaluation was carried out.

(2) Cold start: the cold start problem is a basic challenge of the proposed system and can be divided into a complete cold start and an incomplete cold start. In a full cold start the new user has no interaction, whereas in an incomplete cold start the new user has only a small amount of interaction. In order to evaluate the performance of the model provided by the embodiment of the invention, the recommended persons can be divided into three subtasks: the task is to recommend the existing goods to the new user; the second task is to recommend new articles to the existing users; task three is that w recommends a new item to the new user.

(3) User interaction with the item: let U = { U = ₁ ,u ₂ ,...,u _M And I = { I = } = ₁ ,i ₂ ,...i _N Where U represents a user set, I represents an item set, and M and N represent the number of users and items, respectively, so that a matrix R ∈ R can be used ^MxN To represent interactions between a user and an itemAnd (6) behaviors. The method comprises the following specific steps:

user-item interactions may be converted to a user-item bipartite graph G = (v, epsilon), where v represents a set of nodes, epsilon represents a set of edges in the graph, and | v | = (M + N) represents the number of nodes in the graph; a is an element of R ^MxN Is a contiguous matrix of G.

In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention are further described in detail by the following embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Fig. 2 is a schematic structural diagram illustrating a recommendation system based on task aligned meta learning and augmented graph in an embodiment of the present invention.

It should be understood that TMAG appearing hereinafter in this embodiment is an abbreviation of Task aligned Meta-isolated Augmented Graph, that is, it means learning and Augmented Graph based on Task aligned Meta.

In this embodiment, the recommendation system based on task alignment meta-learning and augmented graph mainly includes three modules: the system comprises a task alignment constructor module, an augmented graph neural network module and a contrast regularization module. The structure and function of these three modules will be explained in detail below.

The task alignment constructor module is used to extract attribute embedding and construct tasks to capture potential cluster knowledge. The task alignment constructor module in turn includes a self-encoding sub-module and a task construction sub-module.

The self-encoding sub-module in the task alignment constructor module includes an attribute-oriented self-encoder, the structure of which refers to part (a) of fig. 2. The attribute-oriented self-encoder takes the user portrait and the article content as input parameters and outputs corresponding user characteristic embedding and article characteristic embedding.

Specifically, the input parameters are first projected into a low-dimensional latent embeddingAnd then reconstructing the input content in the output space after extraction from the potential embedding. E.g. encoder E in the autoencoder _i User portrait as input parameter and output potential item embedding z _i Encoder E _u Using item content as input parameter and outputting potential user embedding z _u (ii) a Potential item embedding z _i Input decoder D _i And outputting the reconstructed input content; potential user embedding z _u Input decoder D _u And outputs the reconstructed input content. It is understood that the user representation is used to describe basic information of the user, such as age, gender, region, income, marital, family, occupation, income, assets, etc.; item content is used to describe item attributes such as actors or genres.

User content input x for user u may be defined _u And item content input x for item i _i The following were used:

x _u ＝[c _u1 ；...；c _up ]；x _i ＝[c _i1 ；...；c _ip ](ii) a Formula (2)

Where p is the number of user content fields, c _up Is d belonging to user u _p Wiki or muti-hot vectors, for content P e { 1. Similarly, Q is the number of item content fields, c _ip Is d belonging to item i _q Wiki or multi-hot vectors, for content Q ∈ { 1.

For ease of understanding, the following text is presented with the output user image x reconstructed using an auto-encoder _u The following are illustrated by way of example:

in this embodiment, a rectifying linear unit (RELU) is selected as the nonlinear activation function to obtain the potential user embedding z _u As follows:

wherein the content of the first and second substances,

and

the parameters are trainable and represent weight and bias, respectively, with superscripts representing the number of layers and subscripts representing user portrait parameters.

It is noted that the Convolutional Neural Network (CNN) is not used to encode the user content in this embodiment, because it is considered that the aggregation of too much information from the neighbors may affect the expressive ability of the user attributes of the alignment task in meta-learning, so the embodiment of the present invention designs a decoder to reconstruct the user portrait x _u As follows:

wherein the content of the first and second substances,

and

the parameters are trainable and represent weight and bias, respectively, with superscripts representing the number of layers and subscripts representing user portrait parameters.

In this embodiment, the objective function of the user attribute-oriented self-encoder is as follows:

wherein, W _u Representing all trainable model parameters; lambda control L ₂ The strength is normalized to prevent overfitting. Similarly, a potential item embedding z can be obtained with a new item attribute oriented self-encoder _i And reconstruct the item content x _i 。

And the task construction sub-module in the task alignment constructor module is used for dividing the users learned from the self-coding sub-module into different user clusters by using a K-Means algorithm, and generating a support set and a query set based on the user clusters to form a task cluster.

In particular, in terms of task construction, learned user and item representations from user-item interaction pairs are not sufficient, since in incomplete cold start recommendation scenarios, user interactions with items are rare. When a new user or new item arrives, potential clustering knowledge should be captured between users with similar attributes to obtain a more accurate representation. Thus, embodiments of the present invention build various meta-tasks to share potential cluster knowledge locally.

In particular, the K-Means algorithm is used to derive x from learned attributes-oriented modules _u (U e U) divide all users U into k different user clusters, i.e. C = { C } ₁ ,C ₂ ...,C _k In which C is _k Representing the kth user cluster. Constructing a meta-training support set based on C following meta-learning paradigm

Wherein

Is the user u ∈ C _k A scored portion of the item. For new users in the cold-start scenario, they are assigned to the cluster where the closest cluster center C is located. These new users form a new cluster C', and a meta-test support set S is constructed according to the new cluster C ^te . Collection

Consisting of a subset of items that the new user interacts with. By the same token, a set of meta-training queries Q can be generated ^tr Containing user-item pairs that are modeled as invisible interactions. Q ^tr For accumulating task loss in meta-training, and meta-test query set Q ^te For evaluating the recommendation results in the meta test. The support set and the query set form a task cluster, and the support set and the query set are at each task T _k Are mutually exclusive.

The augmented graph neural network module is used for relieving data sparsity and capturing high-order user-item interaction. The augmented graph neural network module comprises a graph embedding propagation submodule, a graph augmented generator submodule and a model prediction submodule, and the three modules are detailed below:

and the graph embedding and propagating submodule acquires the task clusters from the task constructing submodule to construct a user-article bipartite graph, and executes GCN to capture high-order structure information of the interaction graph by using the interaction of the user in the task as training data.

Specifically, a user-item bipartite graph is constructed after K different tasks are obtained, and the user's interaction in the kth task is used as T _k The GCN is performed to capture the high-order structural information of the interaction graph.

First, a free-embedded vector e is randomly initialized _u ＝R ^d (e _i ∈R ^d ) To represent user u (item i), where d represents the embedding dimension.

Then, a Lightweight Graph Convolution (LGC) is performed from the neighborhood of user u, as follows:

wherein

Represents the embedding of node u (i) in the (l + 1) layer for memorizing messages from the l-th layer neighbors adjacent thereto; n is a radical of hydrogen _u (N _i ) A neighbor set representing node u (i); | N _u ||N _i | represents the neighbor size of node u (i);

and

similarly, the embedding of item i may be obtained through the LGC. The purpose of the graph embedding propagation submodule is to learn more efficient embedding through information propagation of the user-item bipartite graph only.

It should be appreciated that lightweight graph volume (LGC) is a collaborative filtering model that contains only the most basic partial-neighborhood clustering in the GCN. In particular, lightGCN learns user and item embedding by propagating the user and item embedding linearly on a user-item interaction graph and uses the weighted sum of the embedding learned at all layers as the final embedding.

The graph augmentation generator submodule is used for generating potential interaction for the user based on any one of the following strategies:

in the first strategy, the structure of the interaction graph is mined and potential dependencies of user-item pairs that do not occur en route are captured. In particular, graph-based embedding extends interactions similar to the user's existing interactive items. From the point of view of the ocamer razor, the weighted inner product is used to measure the interest of user u in item i:

wherein

Means the last Lth layer of the item j is embedded, W _g The method comprises the steps of capturing matrix parameters of structural information between a user and an article; σ is a sigmoid function. Unlike the ItemCF algorithm, this strategy not only utilizes neighborhood information, but also captures higher order relationships in bipartite graphs. It should be understood that the ItemCF algorithm refers to an item-based collaborative filtering algorithm, which first calculates the similarity between items and then generates a recommendation list to a user according to the similarity of the items and the historical behavior of the user.

In a second strategy, the user is represented by interactive items and potential items are added based on similarity of attributes of the interactive items to the user. The weighted inner product can also be applied to define the similarity between user u and item i as:

wherein W _a Refers to a matrix parameter that captures attribute information between the user and the item; σ is sigmoid functionAnd (4) counting.

In this example, the two strategies are balanced using an over-parameter α in the range of [0.1], which is expressed as follows:

the loss function for training the potential interaction generator is as follows:

Lgen＝||E-A|| ² (ii) a Formula (10)

In this embodiment, the generated edge is added to the adjacency matrix by setting a threshold t:

wherein the content of the first and second substances,

is the generated adjacency matrix; using a contiguous matrix

To augment the interaction, which is beneficial to alleviate cold start problems.

The model prediction submodule is used for embedding and splicing the final L-layer graph of the user u with corresponding attribute embedding so as to realize modeling of the representation in a finer-grained mode; the splicing results are as follows:

wherein the content of the first and second substances,

indicating a splicing operation.

Similarly, the final embedding f of the item i can be obtained _i And adopting the inner product to estimate the preference of the user on the target object:

in this embodiment, the model parameters are optimized using Bayesian Personalized Ranking (BPR) penalties, which are pairwise penalties that encourage setting the prediction score of observed interactions higher than those that it does not, with the prediction penalties defined as follows:

wherein D = { (u, i, j) | (u, i) ∈ D ⁺ ,(u,j)∈D ^- Represents paired training data; d ⁺ Representing the observed interaction, D ^- Represents no observed interaction, and σ represents a sigmoid function.

The contrast regularization module is used for enhancing potential clustering knowledge by adopting task attributes to contrast regularization terms. In particular, optimizing the ranking incentive penalty may effectively capture the relationship between each pair of training samples. Contrast learning is a loss of ranking motivation. As an extension of the information maximization (InfoMax) principle, contrast learning learns the representation by maximizing the Mutual Information (MI), i.e. comparing positive sample pairs with corresponding negative sample pairs. In embodiments of the present invention, the potential attribute embeddings in the same cluster are pulled together while the potential attribute embeddings in different clusters are pushed away from each other.

Therefore, a task-level contrast regularization term embedded based on the attributes is set to enhance the potential clustering knowledge. More specifically, the attributes in the same task are regarded as positive samples and are denoted as { (z) _u ,z _u′ )|u,u′∈C _k And the attributes in different tasks are negative samples and are recorded as

The supervision of positive sample pairs maintains consistency in the same task. Meanwhile, the supervision of the negative sample pairs enhances the discrimination capability of different tasks. Following the contrast design paradigm, contrast regularization terms are set to maximize MI at the task level, as followsThe following steps:

where sim (-) is a function measuring the similarity between two vectors, and τ is the temperature hyperparameter of softmax. Preferably, sim (-) uses cosine similarity, which can achieve good effect in the model of the embodiment of the present invention, but sim (-) is not limited to cosine similarity, and dot product and the like can also be used, which is not limited in this embodiment. In addition, the synthesis in the denominator can be approximated by a small batch of negative examples of training.

In the present embodiment, to improve recommendations in cold start scenarios, a multitask training strategy is utilized to jointly optimize the main civil task (see equation (14)), the interaction generator task (see equation (10)), and the self-supervised learning task (see equation (15)).

L＝L _pre +λ ₁ L _gen +λ ₂ L _MI +λ ₃ ||⊙|| ² (ii) a Formula (16)

Wherein, <' > indicates L _pre And L _gen All of the trainable parameters in (1), and L _MI No additional parameters were added; lambda [ alpha ] ₁ 、λ ₂ 、λ ₃ The weights of the different losses are parameterized.

Following the MAML framework, embodiments of the present invention perform some second-order gradient descent updates during meta-training to obtain initial parameters that are appropriate for the user and the item. The new user and item representations can then be quickly adjusted with only a small amount of interaction. With the k-th task T _k For example, some gradient descent updates, i.e., inner loop updates, may be performed according to the loss function defined in equation (16). For simplicity, to

An update is performed as follows:

wherein, the first and the second end of the pipe are connected with each other,

represents T _k A loss function of (d); α is the inner loop learning rate; theta' _k Representing the new parameters.

During the outer loop, calculating

The initial parameter θ is updated to be:

where β is the extrinsic cycle learning rate.

In order to better illustrate the technical effects of the technical solution of the present invention, the following description will be made in conjunction with experiments.

The experimental setup was performed in the second part of the experiment:

the experiment was performed based on the following three data sets, including:

DBook dataset: this is a widely used book recommendation data set obtained from bean. The books are classified into old books and new books according to the publication year, and the ratio is about 8:2. Due to the lack of time information about the users, 80% of the users were randomly selected as old users and the remaining 20% as new users in the experiment.

MovieLens dataset: this is a widely used benchmark dataset published by GroupLens for movie recommendations, with movie scores published from 1919 to 2000. Movies are divided into old movies (released before 1998) and new movies (released between 1998 and 2000), about 8:2. To designate new users in the movilens dataset, the users are ranked by their first scoring timestamp, with the last 20% of users being considered as new users for movilens.

Yelp dataset: this is a data set from the Yelp service, widely used for recommendations. Users who joined Yelp about 1 day before 5 in 2014 were considered old users, the rest were considered new users. Also, companies are classified as new and old companies based on the date of the first rating. The ratio of new users to old users is about 8:2.

for each data set, the users and items may be divided into two groups according to the first user interaction time and the first item release time: old and new. The old user's feedback on the old item is considered as meta-training data, and the rest as meta-testing data. 10% of them were randomly selected as validation set.

The meta-test data is divided into three tasks: task 1: recommending old articles for the new user; task 2: recommending new items for the old user; task 3: and recommending new articles for the new user. The experiment only adds an additional recall method in the cold start scene so as to learn better initial embedding in new users and articles. Thus, the recommendation of old users and items is not affected and therefore its performance is not reported.

On these 3 data sets, the user's score for the item was unambiguous, with a score range of 1-5, and scores above 3 were considered positive feedback as they were evaluated for implicit feedback. The statistics are summarized in table 1 below, filtering users with interaction times greater than 100 or less than 13. For each user u, 10 interactive items are randomly selected as a query set, and the rest items are used as support sets for researching how TMAG is influenced by the size of the support sets.

Table 1: statistics of preprocessed data sets

The model of the present invention was compared to three different types of baselines, namely, traditional methods (such as MF algorithm and NeuMF algorithm), GNN-based methods (such as NGCF algorithm, graphSAINT algorithm, and LightGCN algorithm), and cold start methods (such as MeLU algorithm, metaHIN algorithm, and CLCRec algorithm). For both the traditional approach and the CNN-based approach, the base model is trained on meta-training data, and is further fine-tuned using a support set from meta-test data to adapt to the cold-start scenario.

It should be noted that the MF algorithm is a target value for item recommendation using the conventional matrix decomposition of BPR loss optimization, and only using direct interaction between the user and the item as an interaction function. The NeuMF algorithm is the most representative collaborative filtering method based on a deep neural network, and the MLP and matrix decomposition are unified into a general framework to learn the embedding of users and articles. The NGCF algorithm is an efficient collaborative filtering method based on GCN, and collaborative signals are captured through propagation map embedding; it explicitly injects the collaborative signal into the embedding process. The GraphSAINT algorithm is a general GNN model that constructs a complete GCN from the sampled subgraph; normalization techniques are proposed to eliminate the bias and sampling algorithms to reduce the variance are proposed. The LightGCN algorithm improves NGCF by discarding feature transitions and non-linear activations, which learns user and item embedding by linear propagation on user-item interaction graphs. The MelU algorithm applies a multi-step gradient updated MAML to alleviate the problem of user cold start; it only considers user-item interactions and features. The MetaHIN algorithm is a meta-learning method, and captures richer semantics through a high-order graph structure by utilizing the rich semantics of HIN. The CLCRec algorithm maximizes mutual information between the cooperative embedding of the user and the item, and furthermore maximizes mutual information between the cooperative embedding and the item characterization.

Cold start recommendation can be considered as top-N recommendation to a user, this experiment uses three widely used ranking indicators to evaluate model performance: recall (Recall), normalized Discount Cumulative Gain (NDCG) and average precision Mean (MAP). Wherein, recall rate (Recall) represents the proportion of the actual interactive item appearing in the top-K list; the normalized cumulative discount yield (NDCG) is a standard ranking indicator that reflects the relevance and location of each recommended item. The average precision average (MAP) is the average of the average precision scores of each user; and accuracy is the proportion of the actual interactive items in the recommendation list. Using K =10 in this experiment, two-tailed unpaired t-test was performed to detect significant differences between TMAG and the optimal baseline.

Performance comparisons were made in the second part of the experiment:

table 2 below shows the top-10 recommended properties on three data sets (R in the table is an abbreviation for Recall, ND is an abbreviation for NDCG, and M is an abbreviation for MAP), from which the following observations can be made:

table 2: comparison of TMAG model with other baselines on top-10 recommendations

^* Indicating that improvement in TMAG over the best baseline was statistically significant (i.e., single sample t-test with p < 0.05)

In the first class of baseline methods, it can be found that the MF algorithm performs slightly lower than the deep learning method NeuMF, highlighting the key role of nonlinear feature interaction between the user and item embedding. However, neither MF nor NeuMF models the generalization capability of new users and new goods, which leads to poor performance in cold start scenarios.

In the second category of baseline methods, GNN-based methods are a great improvement over traditional methods in all datasets and scenarios, especially LightGCN and GraphSAINT. Since the present experiment focuses on the incomplete cold start problem, the GNN-based model can improve the representation of cold start users and items by exploring higher order relationships. On the Yelp dataset, graphSAINT can be observed to be slightly better than LightGCN.

In the third class of baseline methods, CLCRec achieves competitive performance because it captures more information about the cooperative signal by maximizing the mutual information. As for meta-learning based methods, they show superior performance over traditional methods, since the well-designed training process brings personalized parameter initialization for new users. MetaHIN exceeds the MeLU in every scene because it combines a variety of semantics derived from higher order structures such as meta-paths. These methods are inferior to those based on GNN because they ignore higher order graph structures.

It can be seen that the TMAG method consistently outperforms all baseline methods in all scenarios of the data set. For example, TMAG improved the best baseline by 3.53% -4.91%, 1.83-3.72%, and 3.58-6.71% for Recall @10 over the three data sets. The reasons are summarized as follows: 1) The alignment task extracts potential clustering knowledge among similar users, and can quickly adapt to new users. 2) Graph neural networks capture higher-order user-item relationships, while the MeLU and MetaHIN ignore graph structure information in user-item graphs. Interaction augmentation mitigates the effects of sparse interactions. 3) The task-based contrast regularization term enhances the underlying clustering prior knowledge.

In the third part of the experiment, an ablation study was performed to verify the effectiveness of the proposed TMAG in different situations. In the following tests, task1 was used as a default task and its performance was reported, with evaluation indices of recall @10 and ndcg @10.

Regarding the effect of the task on it: to investigate whether TMAG could benefit from task alignment, the experimental results are summarized in table 3 by searching for cluster numbers within {1,10,20,30,40,50 }:

table 3: effect of the number of alignment tasks

Where TMAG-1 represents a misaligned task and treats each user as a single task. Combined analysis of table 2 and fig. 3, the following observations can be made:

(1) Increasing the number of clusters may enhance the recommendation. TMAG with task alignment consistently outperforms TMAG-1 in all cases, which may attribute the improvement to fine grained modeling of similar groups of users.

(2) TMAG-40 works best on the MovieLens dataset and the DBook dataset, while TMAG-30 works best on the Yelp dataset, with the possible willingness that the user-side information on the Yelp dataset is not as diverse as the other two datasets, and therefore does not require too many clusters to extract attribute knowledge.

(3) TMAG is always superior to other methods when the number of clusters of the three datasets is different. It demonstrates the efficiency of TMAG by extracting the potential cluster knowledge and adapting the globally shared meta-knowledge to the potential cluster knowledge that can be quickly adapted to the new user.

Regarding the effect of the self-encoder: two clustering methods are considered: firstly, directly using auxiliary information of users and articles to carry out one-hot coding; the second is to use an auto-encoder to obtain attribute embedding. The results are described in table 4 below: TMAGs with self-encoders perform better, and improvements can be attributed to the efficient learning of attribute representations by the self-encoder.

Table 4: effects of the self-encoder

Effect on interaction augmentation: in TMAGs, graph structure information and attribute information are combined to augment the adjacency matrix of the graph to alleviate the lack of interaction problem. To verify its validity, different settings may be tested by deleting one type of augmentation or both.

Table 5 shows the results. We can observe that the best setup is to use both types of augmentation simultaneously. Deleting either type reduces performance. And graph augmentation is always better than attribute augmentation, indicating that the collaborative signals captured from graph structures are more beneficial for mining potential interactions than the collaborative signals captured from attributes. Both of these augmentation methods outperform the non-augmented variants, suggesting that the interaction generator helps to learn more expressive representations.

Table 5: influence of variants of the augmentation mode

Subscript symbol: TMAG-ga de-map and attribute augmentation TMAG-a de-attributes augmentation,

TMAG-g is augmented by the removal of the map.

Regarding the effect of contrast regularization: the model of the invention was tested in two cases: one is TMAG without a contrast regularization term; the second is TMAG with a contrast regularization term. Table 6 shows the comparison results: TMAG with the contrast regularization term performs better, which indicates that attribute contrast learning in the aspect of tasks enhances potential clustering prior knowledge and enables a model to learn more informative attribute representations.

Table 6: effects of contrast regularization

In the fourth part of the experiment, a model sensitivity test was performed.

The method is characterized in that firstly, the influence of the size of a support set is exerted, interaction is important data recommended by cold start, and the quality of the interaction is influenced by the sparsity of the data. To study the impact of the level of support set sparsity (i.e., data sparsity), the support set size of the interactive item was ranged from 5 to 70. The results are shown in fig. 3, all methods perform better under a larger support set; however, when the support set is reduced, the performance degradation of the TMAG is the smallest of all methods. The recommended performance of TMAG is excellent even with a short support set and performs well regardless of the length of the interaction history. It should be appreciated that the longer the historical interaction, the fewer the number of users on all data sets. Due to the limited sample size, the performance is considered to be unstable.

Secondly, the influence of the augmentation coefficients, and the robustness of various coefficients can be evaluated from their interactively augmented performance. The experiment was performed in a well-constrained setup of three datasets by varying α in equation (9) over the range of {0,0.2,0.4,0.6,0.8,1.0}, with the results shown in fig. 4. It was observed that TMAG could well be generalized to different coefficients, which suggests the effectiveness of the proposed framework. Between these two augmentation strategies, graph augmentation plays a dominant role. When α is 0.8, the model performs best on Dbook and movilens, whereas α is 1 on Yelp dataset.

Fig. 5 is a schematic flow chart illustrating a recommendation method based on task aligned meta learning and augmented graph in an embodiment of the present invention. The recommendation method in the embodiment includes:

step S51: taking the user portrait and the object content as input parameters, and outputting corresponding user characteristic embedding and object characteristic embedding; and dividing users into different user clusters based on the user characteristic embedding and the article characteristic embedding, and generating a support set and a query set according to the user characteristic embedding and the article characteristic embedding to form a task cluster.

Step S52: constructing a user-article bipartite graph based on the task cluster and capturing high-order user-article interaction information; generating potential interaction for the user based on a preset strategy; embedding and splicing the final layer diagram of the user or the article with the corresponding attribute embedding to obtain the final embedding of the user or the article.

Step S53: task attributes are used to enhance the potential clustering knowledge against the regularization terms.

It should be understood that, in the recommendation method based on the task aligned element learning and the augmented graph in this embodiment, an implementation manner is similar to that of the recommendation system based on the task aligned element learning and the augmented graph, and therefore, details are not repeated.

The recommendation method based on task aligned meta learning and augmented graph provided by the embodiment of the present invention may be implemented by a terminal side or a server side, and as for a hardware structure of an electronic terminal, please refer to fig. 6, which is an optional hardware structure schematic diagram of the electronic terminal 600 provided by the embodiment of the present invention, and the terminal 600 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, and the like. The electronic terminal 600 includes: at least one processor 601, memory 602, at least one network interface 604, and a user interface 606. The various components of the device are coupled together by a bus system 605. It will be appreciated that the bus system 605 is used to enable communications among the components connected. The bus system 605 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for the sake of clarity the various buses are labeled as a bus system in figure 6.

The user interface 606 may include, among other things, a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad, or a touch screen.

It will be appreciated that the memory 602 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), which serves as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), synchronous Static Random Access Memory (SSRAM). The described memory for embodiments of the present invention is intended to comprise, without being limited to, these and any other suitable types of memory.

The memory 602 in the embodiments of the present invention is used to store various kinds of data to support the operation of the electronic terminal 600. Examples of such data include: any executable programs for operating on the electronic terminal 600, such as an operating system 6021 and application programs 6022; the operating system 6021 includes various system programs such as a framework layer, a core library layer, a driver layer, and the like for implementing various basic services and processing hardware-based tasks. The application programs 6022 may include various application programs such as a media player (MediaPlayer), a Browser (Browser), and the like for implementing various application services. The recommendation method based on task alignment meta-learning and augmented graph provided by the embodiment of the invention can be included in the application program 6022.

The method disclosed by the above-mentioned embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 601. The Processor 601 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. Processor 601 may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. The general purpose processor 601 may be a microprocessor or any conventional processor or the like. The steps of the method for optimizing the accessories provided by the embodiment of the invention can be directly embodied as the execution of a hardware decoding processor, or the combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium that is located in a memory and that is read by a processor to perform the steps of the method described above in connection with its hardware.

In an exemplary embodiment, the electronic terminal 600 may be used by one or more Application Specific Integrated Circuits (ASICs), DSPs, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs) for performing the aforementioned methods.

Based on the above embodiments, the present application also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a computer, the computer program causes the computer to execute the methods described in the embodiments of the present application.

The computer readable and writable storage medium may include read-only memory, random-access memory, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, a usb disk, a removable hard disk, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable-writable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are intended to be non-transitory, tangible storage media. Disk and disc, as used in this application, includes Compact Disc (CD), laser disc, optical disc, digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

Based on the above embodiments, the present application further provides a chip, where the chip is used to read a computer program stored in a memory, so as to implement the methods described in the embodiments of the present application.

Based on the above embodiments, the present application provides a chip system, which includes a processor and is used to support a computer device to implement the methods described in the embodiments of the present application. In one possible design, the system-on-chip further includes a memory for storing programs and data necessary for the computer device. The chip system may be constituted by a chip, or may include a chip and other discrete devices.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In summary, the present application provides a recommendation system, method, terminal and medium based on task aligned meta learning and augmented graph, and the present invention proposes TMAG to solve the cold start problem at both model level and feature level. On the model level, a task alignment constructor is provided to capture potential clustering knowledge which can be quickly adapted to a new user, so that the local optimal problem is solved; task-level attributes are also employed to enhance the underlying clustering knowledge versus regularization terms. At the aspect of characteristics, the adjacency matrix of the graph is expanded by combining graph structure information and attribute information, so that the data sparsity problem is relieved. Extensive experiments on three real world datasets demonstrate the effectiveness of the model of the present invention in cold start recommendations. Therefore, the application effectively overcomes various defects in the prior art and has high industrial utilization value.

The above embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.

Claims (12)

1. A recommendation system based on task aligned meta-learning and augmented graph, comprising:

the task alignment constructor module is used for taking the user portrait and the object content as input parameters and outputting corresponding user characteristic embedding and object characteristic embedding; dividing users into different user clusters based on user characteristic embedding and article characteristic embedding, and generating a support set and a query set according to the user clusters to form a task cluster;

the augmented graph neural network module is used for constructing a user-article bipartite graph based on the task cluster and capturing high-order user-article interaction information; generating potential interaction for the user based on a preset strategy; embedding the final layer diagram of the user or the article into corresponding attribute embedding and splicing to obtain the final embedding of the user or the article;

and the comparison regularization module is used for enhancing potential clustering knowledge by adopting the task attribute comparison regularization item.

2. The task-aligned meta-learning and augmentation graph-based recommendation system according to claim 1, wherein said task-aligned constructor module comprises a self-encoding sub-module; the self-encoding submodule comprises an attribute-oriented self-encoder; the objective function of the self-encoder comprises:

wherein, W _u Representing all trainable model parameters; lambda control L ₂ Regularize strength to prevent overfitting; x is the number of _u Is the user content input of user u.

3. The task-aligned meta-learning and augmentation graph-based recommendation system according to claim 1, wherein said task-aligned constructor module comprises a task construction sub-module; the task construction sub-module divides the users learned from the self-coding sub-module into different user clusters by using a K-Means algorithm, and generates a support set and a query set based on the user clusters to form a task cluster.

4. The task aligned meta-learning and augmented graph based recommendation system of claim 1, wherein said augmented graph neural network module comprises a graph embedding propagation sub-module, a graph augmentation generator sub-module, a model prediction sub-module.

5. The recommendation system according to claim 4, wherein the graph embedding propagation submodule constructs a user-item bipartite graph after obtaining the task clusters from the task construction submodule, and executes GCN to capture high-order structure information of the interaction graph using the user's interactions in the tasks as training data.

6. The task aligned meta-learning and augmented graph based recommendation system of claim 4, wherein the graph augmentation generator sub-module is configured to generate potential interactions for the user based on any one of the following policies: the strategy is to mine the structure of the interaction graph and capture the potential dependency relationship of the user-item pair which does not appear on the way; and the second strategy is to represent the user by using the interactive item and add the potential item according to the similarity of the interactive item and the attribute of the user.

7. The recommendation system based on task aligned meta-learning and augmented graph of claim 4, wherein the model prediction sub-module is configured to splice together the final L-layer graph embedding of user u and its corresponding attribute embedding for modeling in a finer-grained based manner.

8. The recommendation system based on task aligned meta-learning and augmentation graph according to claim 1, wherein the recommendation system comprises a loss function setting joint optimization as follows:

L＝L _pre +λ ₁ L _gen +λ ₂ L _MI +λ ₃ ||⊙|| ² ；

wherein, <' > indicates L _pre And L _gen All of the trainable parameters in (1), and L _MI No additional parameters were added; lambda [ alpha ] ₁ 、λ ₂ 、λ ₃ Parameterizing weights of different losses; l is a radical of an alcohol _pre Is a predictive loss function; l is _gen Is a loss function for training the potential interaction generator; l is _MI Is a loss function of the regular contrast term.

9. The task-aligned meta-learning and augmentation graph-based recommendation system according to claim 8, wherein an updated support set and query set are obtained by performing a gradient-descent update on the jointly optimized loss function.

10. A recommendation method based on task alignment meta-learning and augmented graph is characterized by comprising the following steps:

taking the user portrait and the article content as input parameters, and outputting corresponding user characteristic embedding and article characteristic embedding; dividing users into different user clusters based on user characteristic embedding and article characteristic embedding, and generating a support set and a query set according to the user clusters to form a task cluster;

constructing a user-article bipartite graph based on the task cluster and capturing high-order user-article interaction information; generating potential interaction for the user based on a preset strategy; embedding the final layer diagram of the user or the article into corresponding attribute embedding and splicing to obtain the final embedding of the user or the article;

and enhancing potential clustering knowledge by adopting task attributes to compare with regular terms.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method for recommendation based on task aligned meta learning and augmented graph of claim 10.

12. An electronic terminal, comprising at least one processor coupled with at least one memory, the at least one processor configured to read a computer program stored by the at least one memory to perform the method of claim 10.

Images