CN114969533A - Sequence recommendation method based on long-term and short-term preference of user - Google Patents
Sequence recommendation method based on long-term and short-term preference of user Download PDFInfo
- Publication number
- CN114969533A CN114969533A CN202210624731.8A CN202210624731A CN114969533A CN 114969533 A CN114969533 A CN 114969533A CN 202210624731 A CN202210624731 A CN 202210624731A CN 114969533 A CN114969533 A CN 114969533A
- Authority
- CN
- China
- Prior art keywords
- user
- term
- short
- long
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a sequence recommendation method based on long-term and short-term preferences of a user.A conventional recommendation model uses a collaborative filtering algorithm to model the potential interests of the user, but the interests of the user are often complex, changeable and time-varying, and a single model cannot accurately model the interest characteristics of the user.
Description
Technical Field
The invention belongs to the technical field of computer recommendation systems, and relates to a sequence recommendation method based on a recurrent neural network.
Background
At present, in the field of recommendation systems, personalized recommendation systems are receiving more attention. The personalized recommendation system generally utilizes the historical behavior data of the user to carry out deep analysis and mining on factors such as the characteristics, the interests and the like of the user, and the factors are used as information or service for matching the requirements of the user. The most important characteristics of the method are that the method can fully adapt to the situations of unclear and complex user requirements, and can utilize historical data of the user to construct a reasonable algorithm and capture the interests of the user, so that articles or information which are interested by the user can be found in massive information for the user to make decisions. As consumers interact with related websites more and more frequently, people rely more and more on push technology. Therefore, how to mine the interest of the user from the historical behavior data of the user and provide an accurate recommendation service for the user becomes a problem to be solved urgently. But similar algorithms tend to ignore the user interest drift problem. In daily life, user interests generally do not maintain a very stable state, especially on some goods with frequent consumption, such as microblogs, music, and e-commerce websites. The time sequence dynamic factor plays an extremely important role in the practical application of the recommendation system, such as Amazon commodity recommendation, Netflix movie and television product recommendation, Google news and video recommendation and other recommendation systems. When they recommend the data, the data is extremely sensitive to the change of the dynamic interest of the user, and the change of the dynamic interest of the user is hidden in the historical behavior sequence data of the user. Therefore, the sequence recommendation system occupies an extremely important position in the development of the recommendation system at the present stage.
The traditional recommendation algorithm is mostly optimized by content-based recommendation and social network-based recommendation, wherein the interaction behaviors of the user on the commodity exist in the form of independent information. In real life, the user's sequence behaviors are related to each other, even cause and effect each other. In a real-world scenario, the shopping behavior of the user typically occurs in a sequential rather than in isolation, the sequential dependencies of interactions typically exist in the transaction data, and the interactions of the user and the items typically occur in a sequential context, different contexts typically resulting in the user interacting with different items, but such dependencies and interactions are not well captured by conventional content-based or collaborative filtering recommendation algorithms. Over time, both the user's preferences and the popularity of the goods are dynamically changing, which is important for analyzing the user's preferences.
The model structure of a Recurrent Neural Network (RNN) is composed of an input part, a hidden layer part, and an output part, usually an order sequence is used as input, and output results are calculated through the hidden layer structure, and the output result of each layer contains the content of the previous layer. In the RNN-based recommendation model, a recommendation system usually uses a long-short term memory network (LSTM) or a gated round robin unit (GRU) to model a user historical behavior sequence, and in consideration of the good effect of the gated round robin unit in sequence recommendation, the invention adopts the gated round robin unit to model the user short-term interest and introduces an attention mechanism to control an update gate of the GRU network.
The attention mechanism is also one of the main methods for processing sequence information, and the core idea of the attention mechanism is to utilize the complete context information to model the internal structure of the sequence, and simultaneously preserve the time sequence characteristics of the information. When processing a complete commodity sequence, the context of the input information at each moment is known, the model can utilize all the information input by the sequence in the decoding process, and the RNN only uses the state at the last moment when calculating the current level output.
Disclosure of Invention
The invention aims to solve the problem that the change of the user interest cannot be effectively captured in the prior art and model the dynamic interest of the user.
In view of the above, the technical scheme adopted by the invention is as follows: a sequence recommendation method based on long-term and short-term preferences of a user is characterized in that a user historical behavior sequence is input into a model, and a recommendation list finally recommended to the user is obtained through an embedding layer, an interest extraction layer and an interest fusion layer, and the method comprises the following steps:
s1, obtaining a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation.
S2, modeling the user' S short-term interests using gated round-robin units, and introducing an attentive mechanism to control the update gates of the GRU network.
S3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in the hidden factor vector space.
And S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
Further, aiming at the item embedding mode of one-hot coding, aiming at the problem that a large amount of high-dimensional data cannot be optimized well and the model performance is influenced, item2vec is adopted for item embedding, and the item embedding method is used To represent a sequence of user interactionsThe embedded vector of (2).For embedding vector Q u Of (1).Indicates that user u isTime and articleInteractive behavior has occurred.
Further, the process of extracting the user short-term interest by the short-term interest module is as follows, and an attention mechanism is introduced to control an update gate of the GRU network, and the embedded vector of the historical behavior sequence and the target item vector need to be input into the attention network first, and the attention network calculates to obtain the relevant weight w of the embedded vector and the target item vector and transmits the relevant weight w to the GRU network. The attention network calculation formula is as follows:
wherein a is t The value is the output weight and represents the correlation degree of the current historical behavior and the target item, and the larger the value is, the higher the correlation of the current historical behavior and the target item is;is a translation vector that projects the hidden layer to the output weights; an inner product operation; beta is the smoothing index used to adjust the softmax function, f (q, p) is the similarity, W a As a weight matrix, q t For GRU network input, p is the target item, b a Is a bias vector.
The AGRU with attention mechanism proposed by the method is different from the traditional GRU in that the attention mechanism is introduced by updating the gate, and the specific calculation is as follows
The method specifically comprises the following steps:
in the formulaIs a refresh gate of attention machine, u' t It is the original update door that is being updated,h′ t-1 is a hidden state in the AGRU network;is a short-term interest of the user.
Further, the long-term interest module models the user's historical behavior sequence using MLP. In this moduleAll users and items are mapped into the same implicit factor space, and each user u and item i is associated with an implicit factor vector, namelyAnd q is i ,Each element in (b) represents the user's preference for the item, q i The implicit characteristics of the item i are included,and q is i Hidden vectors for users and items, respectively. The specific formula is as follows:
in the formula a * ,w * ,b * Activation function, weight matrix and bias function of the first layer of the MLP respectively,is a long-term interest of the user.
Furthermore, the model designs a method, the long and short term interests are aggregated in a self-adaptive mode, and the fused long and short term interests are used for calculating corresponding recommendation results. In this method, the user interest weight depends on the context, and the following formula is derived based on the attention mechanism
In the formula h T Is a mapping from hidden layer to attention weight; ReLU (. cndot.) is an activation function;is a candidate for attention networks;obtaining a user interest sequence of two stages; w and b are the weight and offset vector of the matrix, respectively;
the relevance of a candidate representing the attention network to the interest sequence should be increased when the relevance of the candidate to the interest sequence is high. Alpha is alpha jt In order to output the weight, the weight is output,for similarity, H u In the case of a sequence of target items,is a target item.
The invention uses the softmax function to convert the weight of the candidate item into the probability expression, and gives out how the system distributes the short-term interestAnd long-term interestImpact on the recommendation.
In the formulaIndicating the preferences found at the user's short-term sequence stage,representing long-term preferences of the user, parameters sought by the preceding formulaThe different phase preferences are adjusted to account for the weight in the final prediction. Finally, a final recommendation list p is obtained at this stage u 。
The invention has the following beneficial technical effects:
(1) the invention belongs to the field of sequence recommendation algorithms, and can perform dynamic personalized recommendation on items for users in a recommendation system.
(2) A novel attention-introducing mechanism user short-term interest extraction gated neural network is provided, and the network structure can be used for better dynamically capturing user short-term preferences.
(3) The user long-term and short-term preference aggregation mode introducing the attention mechanism is provided, and the accuracy of the recommendation result can be effectively improved.
Drawings
FIG. 1 is an algorithm flow of a sequence recommendation model based on long-term and short-term preferences of a user according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described in detail and clearly with reference to the accompanying drawings. The described embodiments are only some of the embodiments of the present invention.
Example 1
As shown in fig. 1, the user historical behavior sequence is a user short-term historical behavior sequence and a user long-term historical behavior sequence, and the embedding layer obtains item embedding vector representation through item2 vec; the short-term interest module models the short-term interest of the user through a gating cycle unit introducing an attention mechanism; inputting the long-term interest of a user into an MLP (Multi-level temporal locality) to search high-order features; and inputting the long-term interest and the short-term interest respectively obtained by the long-term interest module and the short-term interest module into a self-adaptive fusion module introducing an attention mechanism for calculation to obtain a final recommendation result. The method specifically comprises the following steps:
s1, obtaining a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation.
S2, modeling the user' S short-term interests using gated round-robin units, and introducing an attentive mechanism to control the update gates of the GRU network.
S3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in the hidden factor vector space.
And S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
In this example, the data set used was MovieLens-1M, a data set collected and processed by the GroupLens research project group of the computer science and engineering system of minnesota university, which included 836478 scoring data for 3628 movies by 6039 users, and each user had a scoring record for at least more than 20 movies. For experimental needs, the scored explicit data is converted into a two-classification standard, so that the implicit data of 0 or 1 is presented to the user, and the user is indicated whether the user has a history of interaction with the movie.
In this example, a random partitioning method was used for the data set, and 70% of the data set was used for training and 30% was used for testing. This example deletes movies that have fewer than 5 interactions in the dataset.
In particular, the short-term preference behavior sequence in the embodiment of the present invention is the last 5 times of the interaction behavior of the user, and the long-term preference is the history behavior sequence of all the user.
The invention provides a sequence recommendation method based on long-term and short-term preference of a user, and a better recommendation result is obtained by performing experiments on a real data set.
Claims (5)
1. A sequence recommendation method based on long-term and short-term preferences of a user is characterized in that a user historical behavior sequence is input into a model, and a recommendation list finally recommended to the user is obtained through an embedding layer, an interest extraction layer and an interest fusion layer, and the method comprises the following steps:
s1, acquiring a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation;
s2, modeling the short-term interest of the user by adopting a gate control circulation unit, and introducing an attention mechanism to control an update gate of the GRU network;
s3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in a hidden factor vector space;
and S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
3. The method of claim 1, wherein the sequence recommendation method based on the long-term and short-term preferences of the user comprises: the attention mechanism is introduced to control an updating gate of the GRU network, an embedded vector of a historical behavior sequence and a target object vector are firstly input into the attention network, and the attention network calculates to obtain a relative weight w of the embedded vector and the target object vector and transmits the relative weight w to the GRU network; the attention network calculation formula is as follows:
wherein a is t The value is the output weight and represents the degree of correlation between the current historical behavior and the target item, and the larger the value is, the higher the correlation between the current historical behavior and the target item is;is a translation vector that projects the hidden layer to the output weights; an inner product operation; β is the smoothing exponent used to adjust the softmax function;
the specific update gate is calculated as follows:
4. The method for recommending sequences based on long-term and short-term preferences of users according to claim 1, wherein: the long-term interest module models the user's historical behavior sequence using MLP, in which all users and items are mapped into the same implicit factor space, and each user u and item i is associated with an implicit factor vector, i.e.And q is i ,Each element in (a) represents a user's preference for an item, q i The implicit characteristics of the item i are included,and q is i The implicit vectors of the user and the item are respectively, and the specific formula is as follows:
5. The method of claim 1, wherein the sequence recommendation method based on the long-term and short-term preferences of the user comprises: aggregating long-term and short-term interests in a self-adaptive mode, calculating corresponding recommendation results by utilizing the fused long-term and short-term interests, wherein the user interest weight depends on the context, and the following formula is obtained based on an attention mechanism
In the formula h T Is a mapping from hidden layer to attention weight; ReLU (-) is an activation function;is a candidate for attention networks;obtaining a user interest sequence of two stages; w and b are the weight and offset vector of the matrix, respectively;
representing the relevance of the candidate items of the attention network in the interest sequence, and converting the weights of the candidate items into probability expressions by using a softmax function to give out how the system distributes short-term interestAnd long-term interestImpact on recommendation results;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210624731.8A CN114969533A (en) | 2022-06-02 | 2022-06-02 | Sequence recommendation method based on long-term and short-term preference of user |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210624731.8A CN114969533A (en) | 2022-06-02 | 2022-06-02 | Sequence recommendation method based on long-term and short-term preference of user |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114969533A true CN114969533A (en) | 2022-08-30 |
Family
ID=82959424
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210624731.8A Pending CN114969533A (en) | 2022-06-02 | 2022-06-02 | Sequence recommendation method based on long-term and short-term preference of user |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114969533A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115858926A (en) * | 2022-11-29 | 2023-03-28 | 杭州电子科技大学 | User-based complex multi-mode interest extraction and modeling sequence recommendation method |
CN116304279A (en) * | 2023-03-22 | 2023-06-23 | 烟台大学 | Active perception method and system for evolution of user preference based on graph neural network |
CN116562992A (en) * | 2023-07-11 | 2023-08-08 | 数据空间研究院 | Method, device and medium for recommending items for modeling uncertainty of new interests of user |
-
2022
- 2022-06-02 CN CN202210624731.8A patent/CN114969533A/en active Pending
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115858926A (en) * | 2022-11-29 | 2023-03-28 | 杭州电子科技大学 | User-based complex multi-mode interest extraction and modeling sequence recommendation method |
CN115858926B (en) * | 2022-11-29 | 2023-09-01 | 杭州电子科技大学 | Sequence recommendation method based on complex multi-mode interest extraction and modeling of user |
CN116304279A (en) * | 2023-03-22 | 2023-06-23 | 烟台大学 | Active perception method and system for evolution of user preference based on graph neural network |
CN116304279B (en) * | 2023-03-22 | 2024-01-26 | 烟台大学 | Active perception method and system for evolution of user preference based on graph neural network |
CN116562992A (en) * | 2023-07-11 | 2023-08-08 | 数据空间研究院 | Method, device and medium for recommending items for modeling uncertainty of new interests of user |
CN116562992B (en) * | 2023-07-11 | 2023-09-29 | 数据空间研究院 | Method, device and medium for recommending items for modeling uncertainty of new interests of user |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114969533A (en) | Sequence recommendation method based on long-term and short-term preference of user | |
WO2021139415A1 (en) | Data processing method and apparatus, computer readable storage medium, and electronic device | |
Lu et al. | Scalable news recommendation using multi-dimensional similarity and Jaccard–Kmeans clustering | |
CN112052387A (en) | Content recommendation method and device and computer readable storage medium | |
CN117836765A (en) | Click prediction based on multimodal hypergraph | |
CN110069713A (en) | A kind of personalized recommendation method based on user's context perception | |
Lu et al. | Multi-trends enhanced dynamic micro-video recommendation | |
Nalla et al. | Watch hours in minutes: Summarizing videos with user intent | |
Li et al. | Session Recommendation Model Based on Context‐Aware and Gated Graph Neural Networks | |
CN115439770A (en) | Content recall method, device, equipment and storage medium | |
Chen et al. | Exploiting aesthetic features in visual contents for movie recommendation | |
CN114817692A (en) | Method, device and equipment for determining recommended object and computer storage medium | |
Sun | Music Individualization Recommendation System Based on Big Data Analysis | |
Lv et al. | Spatio-temporal attention model based on multi-view for social relation understanding | |
Gahier et al. | Cross Domain Recommendation Systems using Deep Learning: A Systematic Literature Review | |
Lu | Design of a music recommendation model on the basis of multilayer attention representation | |
CN112765465B (en) | User map-based recommendation method | |
Ni et al. | An Improved Sequential Recommendation Algorithm based on Short‐Sequence Enhancement and Temporal Self‐Attention Mechanism | |
Xiao et al. | A better understanding of the interaction between users and items by knowledge graph learning for temporal recommendation | |
CN115544352A (en) | Prediction method and system based on multi-view task relation perception meta-learning | |
Li et al. | Attention with Long‐Term Interval‐Based Deep Sequential Learning for Recommendation | |
Shi et al. | Context-aware recommender systems based on item-grain context clustering | |
Roy et al. | Challenges and issues of recommender system for big data applications | |
Lakshmi Chetana et al. | CF-AMVRGO: Collaborative filtering based adaptive moment variance reduction gradient optimizer for movie recommendations | |
Li et al. | A collaborative filtering recommendation method based on TagIEA expert degree model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |