CN110555132A

CN110555132A - Noise reduction self-encoder recommendation method based on attention model

Info

Publication number: CN110555132A
Application number: CN201910742757.0A
Authority: CN
Inventors: 张延华; 王倩雯; 付琼霄; 李萌; 李庆; 陈冰容
Original assignee: Beijing University of Technology
Current assignee: Beijing University of Technology
Priority date: 2019-08-13
Filing date: 2019-08-13
Publication date: 2019-12-10

Abstract

a film recommendation method of a noise reduction self-encoder based on an attention model belongs to the technical field of film recommendation. In the existing recommendation algorithm, the self-encoder recommendation model is easy to realize and widely applied due to high operation speed, but when a scoring matrix is sparse, the recommendation accuracy is greatly reduced, and the attention degree of auxiliary information and the attention degree of a user to watching records are not considered. In order to solve the problems, the method combines an attention model and a noise reduction self-encoder, learns the preference of a user by using the attention model, and integrates the preference of the user into the noise reduction self-encoder to jointly iterate and update parameters, so that the complete score of the user is predicted. The method has obvious improvement on the aspect of the accuracy of the prediction scoring.

Description

Noise reduction self-encoder recommendation method based on attention model

Technical Field

The method belongs to the technical field of film recommendation, and particularly designs a film recommendation method combining an attention model and a noise reduction self-encoder.

Background

Movies gradually become an indispensable entertainment mode in our daily life, but in the face of a huge movie resource library, how to help users to quickly find favorite movies becomes a main problem faced by a recommendation system.

The traditional movie recommendation method is usually to recommend movies according to the search popularity of the movies, but the types of the movies favored by users are different, so that the satisfaction degree of a recommendation system is low. In recent years, deep learning has enjoyed great success in various fields, such as computer vision, translation, and the like. Deep learning is also widely applied to a recommendation system at present, and because the deep learning can capture complex relation among data, a recommendation algorithm based on the deep learning greatly exceeds a common algorithm in terms of performance.

The existing deep learning recommendation system is widely applied to a recommendation algorithm based on an auto-encoder, but most of algorithms based on the auto-encoder do not consider auxiliary information and different attention degrees of users to watching records, so that the recommendation accuracy is still not ideal. In response to these technical deficiencies, the present invention addresses these problems with an attention model and self-encoder.

disclosure of Invention

The technical problem solved by the invention is as follows: the conventional noise reduction self-encoder has low accuracy, and ignores the influence of the watching records and the auxiliary information of the film of the user, namely the occupation, the work, the gender and the category of the film on the recommendation system.

Aiming at the problems, the invention provides a noise reduction self-encoder recommendation method based on an attention model, which comprises the following steps:

Step 1, obtaining the user, the movie information and the score of the user on the movie from the online public data set.

and 2, processing the movie title information by using a convolutional neural network.

And 3, processing the residual information of the user and the movie and respectively converting the residual information into a user vector and a movie vector.

And 4, selecting the movies with the scores of the first ten users as a user preference matrix according to the scores of the users.

and 5, inputting the user preference matrix into the attention model, and finally calculating to obtain the preference characteristic vector of the user.

And 6, inputting the user score into a noise reduction self-encoder, adding a user auxiliary information vector and a user preference characteristic vector in a hidden layer of the noise reduction self-encoder, and obtaining a complete user score after 100 iterations.

and 7, recommending the movies according to the prediction scores.

According to the invention, the preference characteristics of the user are extracted by using the attention model and are fused into the noise reduction self-encoder, so that the user characteristics are more accurate, and the performance of the recommendation algorithm is greatly improved.

Drawings

FIG. 1 is a diagram of a user-assisted information extraction model

FIG. 2 is a diagram of a movie auxiliary information extraction model

FIG. 3 is a diagram illustrating an overall model of a noise reduction auto-encoder recommendation method based on an attention model according to the present invention

Detailed Description

The invention is described in further detail below with reference to the attached drawing figures:

The invention provides a noise reduction self-encoder recommendation system method based on an attention model, which comprises the following steps:

step 1, obtaining the user, the movie information and the score of the user on the movie from the online public data set. There were 6040 users, 3952 movies, and 1000209 scores, all of which were integers, with scores ranging from 1 to 5. The user, movie information and user rating form are as follows.

User ID	gender	age	occupation	Zip-code
					1	F	1	10	48067
2	M	56	16	70072
					3	M	25	15	55117
…	…	…	…	…

TABLE 1. user information

TABLE 2 movie information

User ID	Movie ID	Rating	Timestamp
				1	1193	5	978300760
1	661	3	978302109
				1	914	3	978301968
…	…	…	…

TABLE 3 Scoring data

and 2, processing the movie title by using the convolutional neural network. Removing irrelevant information such as year in the title, converting the words into 32-dimensional vectors, setting the length of each movie title to be 15 words, and filling up the words by using empty vectors if the length of each movie title is insufficient. And inputting the title matrix into a convolutional neural network, and extracting the title feature vector.

And 3, processing other auxiliary information of the user and the movie, such as the gender, age, occupation and movie category of the user, as shown in the figures 1 and 2. Respectively converting F and M in the gender of the user into two 16-dimensional vectors; since there are seven age categories, the age is shifted to 7 16-dimensional vectors; the number of the working categories is 21, and the working categories are converted into 21 16-dimensional vectors; the categories of the movies are 18 in total, the movies are converted into 18 32-dimensional vectors, in order to ensure that the matrix size of each movie category is the same, the movie category length is set to be 18, and the empty vectors are used for supplement.

Inputting user id, work and age information into a full-connection layer to obtain a 200-dimensional user information vector; and inputting the movie id, the category and the title into the full-connection layer to obtain a 200-dimensional movie information vector.

and 4, selecting ten movies with the highest scores from the movies with the minimum number of 20 movies scored by the user to obtain corresponding movie information vectors to form a 10 x 200 movie matrix preferred by the user.

Step 5, as shown in fig. 3, inputting the movie matrix preferred by the user into the attention network, wherein the formula is as follows:

I′_i＝f_tanh(W'I_i+b') (1)

A_i＝f_softmax(I′_iW^A) (2)

I_ρ＝∑A_iI_i (3)

Wherein I _i is the movie vector of the first ten movie vectors scored by the ith user, W ' is the movie weight matrix and is the normal distribution random matrix with the mean value of 0 and the standard deviation of 0.01, b ' is the movie bias matrix, f _tanh is the tanh function, I _i ' is the movie feature vector of the user, W ^A is the movie feature weight matrix and is the normal distribution random matrix with the mean value of 0 and the standard deviation of 0.01, f _softmax is the softmax function, A _i is the attention weight coefficient of the ith movie vector, and I _ρ is the final user preference vector

Step 6, inputting the user score into a noise reduction self-encoder, adding a user auxiliary information vector and a user preference characteristic vector in a hidden layer of the noise reduction self-encoder for joint iteration, wherein the formula is as follows:

X′＝X+N (4)

h＝f_tanh(WX'+b) (5)

h'＝h+W^sS+I_ρ (6)

Y＝f_tanh(W^zh'+b^z) (7)

wherein X is a user scoring vector, N is a noise vector, X 'is a noise scoring vector, W is a weighting matrix for scoring by adding noise, the weighting matrix is a normal distribution random matrix with a mean value of 0 and a standard deviation of 0.01, b is a bias matrix for the scoring vector by adding noise, h is a hidden layer vector, S is a user auxiliary information vector, W ^s is an auxiliary information vector weighting matrix, the weighting matrix is a normal distribution random matrix with a mean value of 0 and a standard deviation of 0.01, h' is a processed hidden layer vector, W ^z, b ^z are a processed weighting matrix and a bias matrix respectively, and Y is a final complete scoring vector.

The cost function is:

L＝||X-Y||² (8)

L is a cost function, i.e., the squared difference of the input score and the output score.

And obtaining the complete score of the user through at least 100 iterations. The parameter optimization method comprises Adam and RMSProp, and the parameter selection range is as follows: the learning rate is 0.00001-0.01, the training step length is 100-500, and the hidden layer node is 100-500. And continuously changing parameter setting, observing the square difference between the input score and the output score, and when the square difference is the minimum value, adopting the parameter optimization method of Adam, wherein the learning rate is 0.00001, the training compensation is 300, and the number of hidden nodes is 200.

And 7, recommending the movies according to the prediction scores.

Experimental simulation

The invention uses a data set of MovieLens 1M, which comprises 6040 users, 3952 movies, and nearly 10 ten thousand scores, wherein each person has at least 20 scores. Part of score data is randomly extracted to be used as a training set for training, and finally the accuracy of the method is higher than that of a common recommendation algorithm. The invention combines the attention model and the noise reduction self-encoder, solves the common cold start problem in the recommendation system, and improves the recommendation capability to a certain extent.

Claims

1. The attention model-based noise reduction self-encoder recommendation method is characterized by comprising the following steps of:

step 1, obtaining a user, movie information and a score of the user for a movie from an online public data set;

Step 2, processing the movie title information by using a convolutional neural network;

Step 3, processing the user and movie information and respectively converting the user and movie information into a user vector and a movie vector;

Step 4, selecting the movies with the top ten scores of each user according to the scores of the users as a user preference matrix;

step 5, inputting the user preference matrix into an attention model, and finally calculating to obtain a preference characteristic vector of the user;

step 6, inputting the user score into a noise reduction self-encoder, adding a user auxiliary information vector and a user preference characteristic vector in a hidden layer of the noise reduction self-encoder, and obtaining a complete user score after at least 100 iterations;

And 7, recommending the movies according to the prediction scores.

2. The attention model noise reduction self-encoder recommendation method according to claim 1, wherein step 3 specifically converts F, M in user gender into two 16-dimensional vectors respectively; since there are seven age categories, the age is shifted to 7 16-dimensional vectors; the number of the working categories is 21, and the working categories are converted into 21 16-dimensional vectors; the categories of the movies are 18 in total, the movies are converted into 18 32-dimensional vectors, in order to ensure that the sizes of the movie category matrixes are the same, the movie category length is set to be 18, and insufficient empty vectors are used for supplement;

3. the attention model denoising auto-encoder recommendation method according to claim 1, wherein step 5 is to input a user preference movie matrix into the attention network so as to obtain a user preference vector, and the formula is as follows:

I′_i＝f_tanh(W'I_i+b') (1)

A_i＝f_softmax(I′_iW^A) (2)

I_ρ＝∑A_iI_i (3)

Wherein I _i is the movie vector of the first ten of the ith user score, W ' is the movie weight matrix, b ' is the movie bias matrix, f _tanh is the tanh function, I ' _i is the user movie feature vector, W ^A is the movie feature weight matrix, f _softmax is the softmax function, A _i is the attention weight coefficient of the ith movie vector, and I _ρ is the final user preference vector.

4. The attention model denoising autoencoder recommendation method according to claim 1, wherein step 6 is specifically to input the user score into the denoising autoencoder, and add the user auxiliary information vector and the user preference feature vector in the hidden layer of the denoising autoencoder for joint iteration, and the formula is as follows:

X′＝X+N (4)

h＝f_tanh(WX'+b) (5)

h'＝h+W^sS+I_ρ (6)

Y＝f_tanh(W^zh'+b^z) (7)

Wherein X is a user scoring vector, N is a noise vector, X 'is a noise scoring vector, W is a noise scoring weight matrix, b is a noise scoring vector bias matrix, h is a hidden layer vector, S is a user auxiliary information vector, W ^s is an auxiliary information vector weight matrix, h' is a processed hidden layer vector, W ^z and b ^z are respectively a processed weight matrix and a bias matrix, and Y is a final complete scoring vector;

The cost function is:

L＝||X-Y||² (8)

L is a cost function, namely the square difference of the input score and the output score;

And obtaining the complete score of the user through at least 100 iterations.