CN110555132A - Noise reduction self-encoder recommendation method based on attention model - Google Patents

Abstract

A movie recommendation method based on an attention model-based denoising autoencoder belongs to the technical field of movie recommendation. In the existing recommendation algorithms, the autoencoder recommendation model is widely used because of its fast calculation speed and easy implementation. However, when the scoring matrix is sparse, the recommendation accuracy will be greatly reduced, and the difference between the auxiliary information and the user's viewing records has not been considered. Attention. In order to solve the above problems, this method combines the attention model with the denoising autoencoder, uses the attention model to learn the user's preference, and incorporates the denoising autoencoder to iteratively update the parameters to predict the user's complete rating. This method has significantly improved the accuracy of predictive scoring.

Description

An Attention Model-Based Denoising Autoencoder Recommendation Method

technical field

The method belongs to the technical field of movie recommendation, and specifically designs a movie recommendation method combining an attention model and a noise reduction autoencoder.

Background technique

Movies have gradually become an indispensable form of entertainment in our daily life, but in the face of a huge movie resource library, how to help users quickly find their favorite movies has become the main problem faced by the recommendation system.

Traditional movie recommendation methods are often based on movie search popularity, but the types of movies that users like are different, resulting in low satisfaction with the recommendation system. In recent years, deep learning has achieved great success in various fields, such as computer vision, translation, etc. Deep learning is also widely used in recommendation systems. Because deep learning can capture complex connections between data, the performance of recommendation algorithms based on deep learning greatly exceeds that of ordinary algorithms.

The most widely used recommendation system based on autoencoder is the most widely used recommendation algorithm based on autoencoder, but most of the algorithms based on autoencoder do not consider auxiliary information and users' different attention levels to viewing records, resulting in unsatisfactory recommendation accuracy . Aiming at these technical defects, the present invention solves these problems with the help of attention model and autoencoder.

Contents of the invention

The technical problem solved by the present invention is: the common noise reduction autoencoder has low accuracy, and ignores the impact of user viewing records and movie auxiliary information, that is, user occupation, work, gender and movie category on the recommendation system.

In view of the above problems, the present invention provides a noise reduction autoencoder recommendation method based on attention model, the steps are as follows:

Step 1. Obtain users, movie information, and user ratings for movies from online public datasets.

Step 2. Use convolutional neural network to process movie title information.

Step 3. Process the remaining information of users and movies, and transform them into user vectors and movie vectors respectively.

Step 4. According to the user ratings, select the top ten movies rated by each user as the user preference matrix.

Step 5. Input the user preference matrix into the attention model, and finally calculate the user's preference feature vector.

Step 6. Input the user rating into the denoising autoencoder, and add the user auxiliary information vector and user preference feature vector in the hidden layer of the denoising autoencoder. After 100 iterations, the complete user rating is obtained.

Step 7. Movie recommendation based on predicted ratings.

The invention uses the attention model to extract the user's preference feature and integrates it into the noise reduction self-encoder, so that the user feature is more accurate and the performance of the recommendation algorithm is greatly improved.

Description of drawings

Figure 1 is a model diagram of user auxiliary information extraction

Figure 2 is a model diagram of movie auxiliary information extraction

Fig. 3 is the overall model of the noise reduction self-encoder recommendation method based on the attention model of the present invention

Detailed ways

Below in conjunction with accompanying drawing, the present invention is described in further detail:

The present invention provides a kind of attention model-based denoising self-encoder recommendation system method, and the method comprises the following steps:

Step 1. Obtain users, movie information, and user ratings for movies from online public datasets. A total of 6040 users, 3952 movies, 1000209 ratings, ratings from 1 to 5, all integers. The user, movie information and user rating form are as follows.

User ID gender age occupancy Zip-code 1 f 1 10 48067 2 m 56 16 70072 3 m 25 15 55117 … … … … …

Table 1. User Information

Table 2. Movie information

User ID Movie ID Rating Timestamp 1 1193 5 978300760 1 661 3 978302109 1 914 3 978301968 … … … …

Table 3. Scoring data

Step 2. Utilize Convolutional Neural Networks to process movie titles. Remove the irrelevant information such as the year in the title, and convert the word into a 32-dimensional vector. The length of each movie title is set to 15 words. If it is insufficient, fill it with an empty vector. Feed the caption matrix into a convolutional neural network to extract caption feature vectors.

Step 3. As shown in Figures 1 and 2, process other auxiliary information of users and movies, such as user gender, age, occupation, and movie category. Convert the F and M in the user's gender into two 16-dimensional vectors; since there are seven age categories, convert the age into seven 16-dimensional vectors; there are 21 job categories, and convert them into 21 16-dimensional vectors; There are 18 categories, which are converted into 18 32-dimensional vectors. In order to ensure that the size of each movie category matrix is the same, the length of the movie category is set to 18, and empty vectors are used to supplement the shortage.

Input the user id, job, and age information into the fully connected layer to obtain a 200-dimensional user information vector; input the movie id, category, and title into the fully connected layer to obtain a 200-dimensional movie information vector.

Step 4. There are at least 20 movies rated by the user, and the ten movies with the highest ratings are selected from them, and the corresponding movie information vectors are obtained to form a 10*200 user preference movie matrix.

Step 5. As shown in Figure 3, input the user preference movie matrix into the attention network, the formula is as follows:

I′ _i =f _tanh (W'I _i +b') (1)

A _i ＝f _softmax (I′ _i W ^A ) (2)

I _ρ ＝∑A _i I _i (3)

Among them, I _i is the top ten movie vectors rated by the i-th user, and W' is the movie weight matrix, which is a normal distribution random matrix with a mean of 0 and a standard deviation of 0.01. b' is a movie bias matrix, f _tanh is a tanh function, I _i ' is a user movie feature vector, W ^A is a movie feature weight matrix, which is a normal distribution random matrix with a mean of 0 and a standard deviation of 0.01. f _softmax is the softmax function, A _i is the attention weight coefficient of the i-th movie vector, I _ρ is the final user preference vector

Step 6. Input the user rating into the denoising self-encoder, and add the user auxiliary information vector and user preference feature vector to the hidden layer of the denoising self-encoder to iterate together. The formula is as follows:

X'=X+N (4)

h＝f _tanh (WX'+b) (5)

h'＝h+W ^s S+I _ρ (6)

Y＝f _tanh (W ^z h'+b ^z ) (7)

Where X is the user rating vector, N is the noise vector, X' is the noise-adding score vector, W is the weight matrix of the noise-adding score, which is a normal distribution random matrix with a mean of 0 and a standard deviation of 0.01, and b is the noise-adding score The offset matrix of the vector, h is the hidden layer vector, S is the user auxiliary information vector, W ^s is the weight matrix of the auxiliary information vector, which is a normal distribution random matrix with a mean of 0 and a standard deviation of 0.01, and h' is the processed The hidden layer vector of , W ^z , b ^z are the processed weight matrix and bias matrix respectively, and Y is the final complete scoring vector.

The cost function is:

L＝||XY|| ² (8)

L is the cost function, which is the squared difference between the input score and the output score.

After at least 100 iterations, complete user ratings are obtained. The parameter optimization methods include Adam and RMSProp, and the parameter selection range is: the learning rate is between 0.00001 and 0.01, the training step is between 100 and 500, and the hidden layer nodes are between 100 and 500. Constantly change the parameter settings and observe the square difference between the input and output scores. When the square difference reaches the minimum value, the parameter optimization method is Adam, the learning rate is 0.00001, the training compensation is 300, and the number of hidden nodes is 200.

Step 7. Movie recommendation based on predicted ratings.

Experimental simulation

The present invention uses the dataset of MovieLens 1M, including 6040 users, 3952 movies, nearly 100,000 ratings, and at least 20 ratings per person. Part of the scoring data is randomly selected as a training set for training, and finally the accuracy of the present invention is higher than that of common recommendation algorithms. The invention combines the attention model with the noise reduction self-encoder, solves the common cold start problem in the recommendation system, and improves the recommendation ability to a certain extent.

Claims (4)

1. The attention model-based noise reduction self-encoder recommendation method is characterized by comprising the following steps of:

step 1, obtaining a user, movie information and a score of the user for a movie from an online public data set;

Step 2, processing the movie title information by using a convolutional neural network;

Step 3, processing the user and movie information and respectively converting the user and movie information into a user vector and a movie vector;

Step 4, selecting the movies with the top ten scores of each user according to the scores of the users as a user preference matrix;

step 5, inputting the user preference matrix into an attention model, and finally calculating to obtain a preference characteristic vector of the user;

step 6, inputting the user score into a noise reduction self-encoder, adding a user auxiliary information vector and a user preference characteristic vector in a hidden layer of the noise reduction self-encoder, and obtaining a complete user score after at least 100 iterations;

And 7, recommending the movies according to the prediction scores.

2. The attention model noise reduction self-encoder recommendation method according to claim 1, wherein step 3 specifically converts F, M in user gender into two 16-dimensional vectors respectively; since there are seven age categories, the age is shifted to 7 16-dimensional vectors; the number of the working categories is 21, and the working categories are converted into 21 16-dimensional vectors; the categories of the movies are 18 in total, the movies are converted into 18 32-dimensional vectors, in order to ensure that the sizes of the movie category matrixes are the same, the movie category length is set to be 18, and insufficient empty vectors are used for supplement;

inputting user id, work and age information into a full-connection layer to obtain a 200-dimensional user information vector; and inputting the movie id, the category and the title into the full-connection layer to obtain a 200-dimensional movie information vector.

3. the attention model denoising auto-encoder recommendation method according to claim 1, wherein step 5 is to input a user preference movie matrix into the attention network so as to obtain a user preference vector, and the formula is as follows:

I′_i＝f_tanh(W'I_i+b') (1)

A_i＝f_softmax(I′_iW^A) (2)

I_ρ＝∑A_iI_i (3)

Wherein I _i is the movie vector of the first ten of the ith user score, W ' is the movie weight matrix, b ' is the movie bias matrix, f _tanh is the tanh function, I ' _i is the user movie feature vector, W ^A is the movie feature weight matrix, f _softmax is the softmax function, A _i is the attention weight coefficient of the ith movie vector, and I _ρ is the final user preference vector.

4. The attention model denoising autoencoder recommendation method according to claim 1, wherein step 6 is specifically to input the user score into the denoising autoencoder, and add the user auxiliary information vector and the user preference feature vector in the hidden layer of the denoising autoencoder for joint iteration, and the formula is as follows:

X′＝X+N (4)

h＝f_tanh(WX'+b) (5)

h'＝h+W^sS+I_ρ (6)

Y＝f_tanh(W^zh'+b^z) (7)

Wherein X is a user scoring vector, N is a noise vector, X 'is a noise scoring vector, W is a noise scoring weight matrix, b is a noise scoring vector bias matrix, h is a hidden layer vector, S is a user auxiliary information vector, W ^s is an auxiliary information vector weight matrix, h' is a processed hidden layer vector, W ^z and b ^z are respectively a processed weight matrix and a bias matrix, and Y is a final complete scoring vector;

The cost function is:

L＝||X-Y||² (8)

L is a cost function, namely the square difference of the input score and the output score;

And obtaining the complete score of the user through at least 100 iterations.