CN114969533A - Sequence recommendation method based on long-term and short-term preference of user - Google Patents

Sequence recommendation method based on long-term and short-term preference of user Download PDF

Info

Publication number
CN114969533A
CN114969533A CN202210624731.8A CN202210624731A CN114969533A CN 114969533 A CN114969533 A CN 114969533A CN 202210624731 A CN202210624731 A CN 202210624731A CN 114969533 A CN114969533 A CN 114969533A
Authority
CN
China
Prior art keywords
user
term
short
long
interest
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210624731.8A
Other languages
Chinese (zh)
Inventor
刘伯红
石翰睿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202210624731.8A priority Critical patent/CN114969533A/en
Publication of CN114969533A publication Critical patent/CN114969533A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a sequence recommendation method based on long-term and short-term preferences of a user.A conventional recommendation model uses a collaborative filtering algorithm to model the potential interests of the user, but the interests of the user are often complex, changeable and time-varying, and a single model cannot accurately model the interest characteristics of the user.

Description

Sequence recommendation method based on long-term and short-term preference of user
Technical Field
The invention belongs to the technical field of computer recommendation systems, and relates to a sequence recommendation method based on a recurrent neural network.
Background
At present, in the field of recommendation systems, personalized recommendation systems are receiving more attention. The personalized recommendation system generally utilizes the historical behavior data of the user to carry out deep analysis and mining on factors such as the characteristics, the interests and the like of the user, and the factors are used as information or service for matching the requirements of the user. The most important characteristics of the method are that the method can fully adapt to the situations of unclear and complex user requirements, and can utilize historical data of the user to construct a reasonable algorithm and capture the interests of the user, so that articles or information which are interested by the user can be found in massive information for the user to make decisions. As consumers interact with related websites more and more frequently, people rely more and more on push technology. Therefore, how to mine the interest of the user from the historical behavior data of the user and provide an accurate recommendation service for the user becomes a problem to be solved urgently. But similar algorithms tend to ignore the user interest drift problem. In daily life, user interests generally do not maintain a very stable state, especially on some goods with frequent consumption, such as microblogs, music, and e-commerce websites. The time sequence dynamic factor plays an extremely important role in the practical application of the recommendation system, such as Amazon commodity recommendation, Netflix movie and television product recommendation, Google news and video recommendation and other recommendation systems. When they recommend the data, the data is extremely sensitive to the change of the dynamic interest of the user, and the change of the dynamic interest of the user is hidden in the historical behavior sequence data of the user. Therefore, the sequence recommendation system occupies an extremely important position in the development of the recommendation system at the present stage.
The traditional recommendation algorithm is mostly optimized by content-based recommendation and social network-based recommendation, wherein the interaction behaviors of the user on the commodity exist in the form of independent information. In real life, the user's sequence behaviors are related to each other, even cause and effect each other. In a real-world scenario, the shopping behavior of the user typically occurs in a sequential rather than in isolation, the sequential dependencies of interactions typically exist in the transaction data, and the interactions of the user and the items typically occur in a sequential context, different contexts typically resulting in the user interacting with different items, but such dependencies and interactions are not well captured by conventional content-based or collaborative filtering recommendation algorithms. Over time, both the user's preferences and the popularity of the goods are dynamically changing, which is important for analyzing the user's preferences.
The model structure of a Recurrent Neural Network (RNN) is composed of an input part, a hidden layer part, and an output part, usually an order sequence is used as input, and output results are calculated through the hidden layer structure, and the output result of each layer contains the content of the previous layer. In the RNN-based recommendation model, a recommendation system usually uses a long-short term memory network (LSTM) or a gated round robin unit (GRU) to model a user historical behavior sequence, and in consideration of the good effect of the gated round robin unit in sequence recommendation, the invention adopts the gated round robin unit to model the user short-term interest and introduces an attention mechanism to control an update gate of the GRU network.
The attention mechanism is also one of the main methods for processing sequence information, and the core idea of the attention mechanism is to utilize the complete context information to model the internal structure of the sequence, and simultaneously preserve the time sequence characteristics of the information. When processing a complete commodity sequence, the context of the input information at each moment is known, the model can utilize all the information input by the sequence in the decoding process, and the RNN only uses the state at the last moment when calculating the current level output.
Disclosure of Invention
The invention aims to solve the problem that the change of the user interest cannot be effectively captured in the prior art and model the dynamic interest of the user.
In view of the above, the technical scheme adopted by the invention is as follows: a sequence recommendation method based on long-term and short-term preferences of a user is characterized in that a user historical behavior sequence is input into a model, and a recommendation list finally recommended to the user is obtained through an embedding layer, an interest extraction layer and an interest fusion layer, and the method comprises the following steps:
s1, obtaining a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation.
S2, modeling the user' S short-term interests using gated round-robin units, and introducing an attentive mechanism to control the update gates of the GRU network.
S3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in the hidden factor vector space.
And S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
Further, aiming at the item embedding mode of one-hot coding, aiming at the problem that a large amount of high-dimensional data cannot be optimized well and the model performance is influenced, item2vec is adopted for item embedding, and the item embedding method is used
Figure BDA0003676503580000021
Figure BDA0003676503580000022
To represent a sequence of user interactions
Figure BDA0003676503580000023
The embedded vector of (2).
Figure BDA0003676503580000024
For embedding vector Q u Of (1).
Figure BDA0003676503580000025
Indicates that user u is
Figure BDA0003676503580000026
Time and article
Figure BDA0003676503580000027
Interactive behavior has occurred.
Further, the process of extracting the user short-term interest by the short-term interest module is as follows, and an attention mechanism is introduced to control an update gate of the GRU network, and the embedded vector of the historical behavior sequence and the target item vector need to be input into the attention network first, and the attention network calculates to obtain the relevant weight w of the embedded vector and the target item vector and transmits the relevant weight w to the GRU network. The attention network calculation formula is as follows:
Figure BDA0003676503580000028
Figure BDA0003676503580000029
wherein a is t The value is the output weight and represents the correlation degree of the current historical behavior and the target item, and the larger the value is, the higher the correlation of the current historical behavior and the target item is;
Figure BDA00036765035800000210
is a translation vector that projects the hidden layer to the output weights; an inner product operation; beta is the smoothing index used to adjust the softmax function, f (q, p) is the similarity, W a As a weight matrix, q t For GRU network input, p is the target item, b a Is a bias vector.
The AGRU with attention mechanism proposed by the method is different from the traditional GRU in that the attention mechanism is introduced by updating the gate, and the specific calculation is as follows
The method specifically comprises the following steps:
Figure BDA0003676503580000031
Figure BDA0003676503580000032
in the formula
Figure BDA0003676503580000038
Is a refresh gate of attention machine, u' t It is the original update door that is being updated,
Figure BDA0003676503580000039
h′ t-1 is a hidden state in the AGRU network;
Figure BDA00036765035800000310
is a short-term interest of the user.
Further, the long-term interest module models the user's historical behavior sequence using MLP. In this moduleAll users and items are mapped into the same implicit factor space, and each user u and item i is associated with an implicit factor vector, namely
Figure BDA00036765035800000311
And q is i ，
Figure BDA00036765035800000312
Each element in (b) represents the user's preference for the item, q i The implicit characteristics of the item i are included,
Figure BDA00036765035800000313
and q is i Hidden vectors for users and items, respectively. The specific formula is as follows:
Figure BDA0003676503580000033
in the formula a * ，w * ，b * Activation function, weight matrix and bias function of the first layer of the MLP respectively,
Figure BDA00036765035800000314
is a long-term interest of the user.
Furthermore, the model designs a method, the long and short term interests are aggregated in a self-adaptive mode, and the fused long and short term interests are used for calculating corresponding recommendation results. In this method, the user interest weight depends on the context, and the following formula is derived based on the attention mechanism
Figure BDA0003676503580000034
In the formula h T Is a mapping from hidden layer to attention weight; ReLU (. cndot.) is an activation function;
Figure BDA00036765035800000315
is a candidate for attention networks;
Figure BDA00036765035800000316
obtaining a user interest sequence of two stages; w and b are the weight and offset vector of the matrix, respectively;
Figure BDA0003676503580000035
Figure BDA0003676503580000036
Figure BDA00036765035800000317
the relevance of a candidate representing the attention network to the interest sequence should be increased when the relevance of the candidate to the interest sequence is high. Alpha is alpha jt In order to output the weight, the weight is output,
Figure BDA00036765035800000318
for similarity, H u In the case of a sequence of target items,
Figure BDA00036765035800000319
is a target item.
The invention uses the softmax function to convert the weight of the candidate item into the probability expression, and gives out how the system distributes the short-term interest
Figure BDA00036765035800000320
And long-term interest
Figure BDA00036765035800000321
Impact on the recommendation.
Figure BDA0003676503580000037
In the formula
Figure BDA00036765035800000322
Indicating the preferences found at the user's short-term sequence stage,
Figure BDA00036765035800000323
representing long-term preferences of the user, parameters sought by the preceding formula
Figure BDA00036765035800000324
The different phase preferences are adjusted to account for the weight in the final prediction. Finally, a final recommendation list p is obtained at this stage u 。
The invention has the following beneficial technical effects:
(1) the invention belongs to the field of sequence recommendation algorithms, and can perform dynamic personalized recommendation on items for users in a recommendation system.
(2) A novel attention-introducing mechanism user short-term interest extraction gated neural network is provided, and the network structure can be used for better dynamically capturing user short-term preferences.
(3) The user long-term and short-term preference aggregation mode introducing the attention mechanism is provided, and the accuracy of the recommendation result can be effectively improved.
Drawings
FIG. 1 is an algorithm flow of a sequence recommendation model based on long-term and short-term preferences of a user according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described in detail and clearly with reference to the accompanying drawings. The described embodiments are only some of the embodiments of the present invention.
Example 1
As shown in fig. 1, the user historical behavior sequence is a user short-term historical behavior sequence and a user long-term historical behavior sequence, and the embedding layer obtains item embedding vector representation through item2 vec; the short-term interest module models the short-term interest of the user through a gating cycle unit introducing an attention mechanism; inputting the long-term interest of a user into an MLP (Multi-level temporal locality) to search high-order features; and inputting the long-term interest and the short-term interest respectively obtained by the long-term interest module and the short-term interest module into a self-adaptive fusion module introducing an attention mechanism for calculation to obtain a final recommendation result. The method specifically comprises the following steps:
s1, obtaining a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation.
S2, modeling the user' S short-term interests using gated round-robin units, and introducing an attentive mechanism to control the update gates of the GRU network.
S3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in the hidden factor vector space.
And S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
In this example, the data set used was MovieLens-1M, a data set collected and processed by the GroupLens research project group of the computer science and engineering system of minnesota university, which included 836478 scoring data for 3628 movies by 6039 users, and each user had a scoring record for at least more than 20 movies. For experimental needs, the scored explicit data is converted into a two-classification standard, so that the implicit data of 0 or 1 is presented to the user, and the user is indicated whether the user has a history of interaction with the movie.
In this example, a random partitioning method was used for the data set, and 70% of the data set was used for training and 30% was used for testing. This example deletes movies that have fewer than 5 interactions in the dataset.
In particular, the short-term preference behavior sequence in the embodiment of the present invention is the last 5 times of the interaction behavior of the user, and the long-term preference is the history behavior sequence of all the user.
The invention provides a sequence recommendation method based on long-term and short-term preference of a user, and a better recommendation result is obtained by performing experiments on a real data set.

Claims (5)

1. A sequence recommendation method based on long-term and short-term preferences of a user is characterized in that a user historical behavior sequence is input into a model, and a recommendation list finally recommended to the user is obtained through an embedding layer, an interest extraction layer and an interest fusion layer, and the method comprises the following steps:
s1, acquiring a user historical behavior sequence, and introducing item2vec to embed items to generate user vector representation;
s2, modeling the short-term interest of the user by adopting a gate control circulation unit, and introducing an attention mechanism to control an update gate of the GRU network;
s3, modeling the long-term interest of the user by using MLP, and searching high-order features of the user in a hidden factor vector space;
and S4, aggregating the long-term interest and the short-term interest based on an attention mechanism, and calculating a corresponding recommendation result by using the fused long-term interest and the fused short-term interest.
2. The method of claim 1, wherein the sequence recommendation method based on the long-term and short-term preferences of the user comprises: item embedding with the item2vec, used herein
Figure FDA0003676503570000011
To represent a sequence of user interactions
Figure FDA0003676503570000012
The embedded vector of (2).
3. The method of claim 1, wherein the sequence recommendation method based on the long-term and short-term preferences of the user comprises: the attention mechanism is introduced to control an updating gate of the GRU network, an embedded vector of a historical behavior sequence and a target object vector are firstly input into the attention network, and the attention network calculates to obtain a relative weight w of the embedded vector and the target object vector and transmits the relative weight w to the GRU network; the attention network calculation formula is as follows:
Figure FDA0003676503570000013
Figure FDA0003676503570000014
wherein a is t The value is the output weight and represents the degree of correlation between the current historical behavior and the target item, and the larger the value is, the higher the correlation between the current historical behavior and the target item is;
Figure FDA0003676503570000015
is a translation vector that projects the hidden layer to the output weights; an inner product operation; β is the smoothing exponent used to adjust the softmax function;
the specific update gate is calculated as follows:
Figure FDA0003676503570000016
Figure FDA0003676503570000017
in the formula
Figure FDA0003676503570000018
Is a refresh gate of attention machine, u' t It is the original update door that is being updated,
Figure FDA0003676503570000019
h′ t-1 is a hidden state in the AGRU network;
Figure FDA00036765035700000110
is a short-term interest of the user.
4. The method for recommending sequences based on long-term and short-term preferences of users according to claim 1, wherein: the long-term interest module models the user's historical behavior sequence using MLP, in which all users and items are mapped into the same implicit factor space, and each user u and item i is associated with an implicit factor vector, i.e.
Figure FDA00036765035700000111
And q is i ，
Figure FDA00036765035700000112
Each element in (a) represents a user's preference for an item, q i The implicit characteristics of the item i are included,
Figure FDA00036765035700000113
and q is i The implicit vectors of the user and the item are respectively, and the specific formula is as follows:
Figure FDA00036765035700000114
in the formula a * ，w * ，b * Activation function, weight matrix and bias function of the first layer of the MLP respectively,
Figure FDA00036765035700000115
is a long-term interest of the user.
5. The method of claim 1, wherein the sequence recommendation method based on the long-term and short-term preferences of the user comprises: aggregating long-term and short-term interests in a self-adaptive mode, calculating corresponding recommendation results by utilizing the fused long-term and short-term interests, wherein the user interest weight depends on the context, and the following formula is obtained based on an attention mechanism
Figure FDA0003676503570000021
In the formula h T Is a mapping from hidden layer to attention weight; ReLU (-) is an activation function;
Figure FDA0003676503570000022
is a candidate for attention networks;
Figure FDA0003676503570000023
obtaining a user interest sequence of two stages; w and b are the weight and offset vector of the matrix, respectively;
Figure FDA0003676503570000024
Figure FDA0003676503570000025
Figure FDA0003676503570000026
representing the relevance of the candidate items of the attention network in the interest sequence, and converting the weights of the candidate items into probability expressions by using a softmax function to give out how the system distributes short-term interest
Figure FDA0003676503570000027
And long-term interest
Figure FDA0003676503570000028
Impact on recommendation results;
Figure FDA0003676503570000029
in the formula
Figure FDA00036765035700000210
Indicating the preferences found at the user's short-term sequence stage,
Figure FDA00036765035700000211
representing the user's long-term preferences, at which stage the final recommendation list is obtained.
CN202210624731.8A 2022-06-02 2022-06-02 Sequence recommendation method based on long-term and short-term preference of user Pending CN114969533A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210624731.8A CN114969533A (en) 2022-06-02 2022-06-02 Sequence recommendation method based on long-term and short-term preference of user

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210624731.8A CN114969533A (en) 2022-06-02 2022-06-02 Sequence recommendation method based on long-term and short-term preference of user

Publications (1)

Publication Number Publication Date
CN114969533A true CN114969533A (en) 2022-08-30

Family

ID=82959424

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210624731.8A Pending CN114969533A (en) 2022-06-02 2022-06-02 Sequence recommendation method based on long-term and short-term preference of user

Country Status (1)

Country Link
CN (1) CN114969533A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858926A (en) * 2022-11-29 2023-03-28 杭州电子科技大学 User-based complex multi-mode interest extraction and modeling sequence recommendation method
CN116304279A (en) * 2023-03-22 2023-06-23 烟台大学 Active perception method and system for evolution of user preference based on graph neural network
CN116562992A (en) * 2023-07-11 2023-08-08 数据空间研究院 Method, device and medium for recommending items for modeling uncertainty of new interests of user

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115858926A (en) * 2022-11-29 2023-03-28 杭州电子科技大学 User-based complex multi-mode interest extraction and modeling sequence recommendation method
CN115858926B (en) * 2022-11-29 2023-09-01 杭州电子科技大学 Sequence recommendation method based on complex multi-mode interest extraction and modeling of user
CN116304279A (en) * 2023-03-22 2023-06-23 烟台大学 Active perception method and system for evolution of user preference based on graph neural network
CN116304279B (en) * 2023-03-22 2024-01-26 烟台大学 Active perception method and system for evolution of user preference based on graph neural network
CN116562992A (en) * 2023-07-11 2023-08-08 数据空间研究院 Method, device and medium for recommending items for modeling uncertainty of new interests of user
CN116562992B (en) * 2023-07-11 2023-09-29 数据空间研究院 Method, device and medium for recommending items for modeling uncertainty of new interests of user

Similar Documents

Publication Publication Date Title
CN114969533A (en) Sequence recommendation method based on long-term and short-term preference of user
US11188830B2 (en) Method and system for user profiling for content recommendation
WO2021139415A1 (en) Data processing method and apparatus, computer readable storage medium, and electronic device
Lu et al. Scalable news recommendation using multi-dimensional similarity and Jaccard–Kmeans clustering
CN112052387A (en) Content recommendation method and device and computer readable storage medium
CN117836765A (en) Click prediction based on multimodal hypergraph
Wang et al. Interval-enhanced graph transformer solution for session-based recommendation
Nalla et al. Watch hours in minutes: Summarizing videos with user intent
CN114817692A (en) Method, device and equipment for determining recommended object and computer storage medium
Sun Music Individualization Recommendation System Based on Big Data Analysis
Chen et al. Exploiting aesthetic features in visual contents for movie recommendation
Lv et al. Spatio-temporal attention model based on multi-view for social relation understanding
Xiao et al. A better understanding of the interaction between users and items by knowledge graph learning for temporal recommendation
Li et al. Attention with long-term interval-based deep sequential learning for recommendation
Gahier et al. Cross Domain Recommendation Systems using Deep Learning: A Systematic Literature Review
Shi et al. Context-aware recommender systems based on item-grain context clustering
Li et al. A collaborative filtering recommendation method based on TagIEA expert degree model
CN115439770A (en) Content recall method, device, equipment and storage medium
Roy et al. Challenges and issues of recommender system for big data applications
Li et al. Disentangled-feature and composite-prior VAE on social recommendation for new users
Lakshmi Chetana et al. CF-AMVRGO: Collaborative Filtering based Adaptive Moment Variance Reduction Gradient Optimizer for Movie Recommendations
Li et al. Multi-hypergraph Neural Network with Fusion of Location Information for Session-based Recommendation.
Yuan et al. Modeling Long-and Short-Term Service Recommendations with a Deep Multi-Interest Network for Edge Computing
CN115470397B (en) Content recommendation method, device, computer equipment and storage medium
Zhang [Retracted] Design and Implementation of Sports Industry Information Service Management System Based on Data Mining

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination