CN102930341A - Optimal training method of collaborative filtering recommendation model - Google Patents

Abstract

The invention discloses an optimal training method of a collaborative filtering recommendation model, belonging to the technical field of data mining and individual recommendation. The optimal training method comprises the following steps of: arranging hidden characteristic matrixes seperately, so as to eliminate the interdependency of user hidden characteristics and item hidden characteristics in a training process; secondly, dividing the hidden characteristic matrixes into a user hidden characteristic training process and an item hidden characteristic training process which separate stochastic gradient descent; and finally, executing the user hidden characteristic training process and the item hidden characteristic training process. With the adoption of the optimal training method provided by the invention, the interdependency of the user hidden characteristic matrixes and the item hidden characteristic matrixes can be eliminated by optimizing a training process of the collaborative filtering recommendation model, so that the expandability is improved; the rate of convergence is fast; the training round number desired by the convergence is less; and the establishment speed of the recommendation model can be improved.

Description

A kind of optimization training method of collaborative filtering recommending model

Technical field

The invention belongs to data mining and personalized recommendation technical field, particularly relate to a kind of optimization training method of collaborative filtering recommending model.

Background technology

The explosive increase of internet information scale brings the problem of information overload, and excess of information presents simultaneously, so that the user is difficult to therefrom filter out the effective part to the individual, information utilization reduces on the contrary.The personalized recommendation technology is the important branch of data mining research field, and target is the Intelligent Service that " information is looked for the people " is provided by setting up personalized recommendation system, fundamentally to solve information overload.

As recommending the generation source, recommended models is the core component in the personalized recommendation system, and possess good recommendation accuracy rate and extensibility based on the recommended models of matrix factorization because of it, it is the widely used recommended models of a class, at present, the personalized recommendation technology can be according to user's historical behavior, inner link to user and information is analyzed, thereby user and its may be got up in interested informational linkage, provide information to look for people's Intelligent Service, thereby solve the problem of information overload, recommended models is the emphasis of personalized recommendation technical field research as the core component in the commending system, because matrix factorization recommended models possesses very high recommendation accuracy rate and good extensibility, thereby possesses widely range of application, but, because in the training process of matrix factorization recommended models, the hidden feature of its user and the hidden feature of project interdepend, can't carry out parallelization, thereby limit the further popularization of model.

Therefore on the basis of further investigation based on the recommended models training process of matrix factorization, those skilled in the art are devoted to develop a kind of optimization training method of collaborative filtering recommending model, can eliminate the hidden eigenmatrix of user and the relation of interdependence of the hidden eigenmatrix of project in training process in the matrix factorization recommended models.

Summary of the invention

Because the defects of prior art, technical matters to be solved by this invention provides a kind of optimization training method that can eliminate the hidden eigenmatrix of user and the hidden eigenmatrix of project complementary collaborative filtering recommending model in training process in the matrix factorization recommended models.

By the training process of matrix factorization recommended models is analyzed, determine that the interdependent property of the hidden feature of user and project in training process is to be caused by the adduction relationship to the state value of corresponding hidden feature, therefore, by introducing single-row random gradient down machine system, eliminated the hidden feature of user and project quoting corresponding hidden eigenstate value in training process, thereby the hidden eigenmatrix of user and the interdependent property of the hidden eigenmatrix of project in training process have been eliminated, again based on this, training process to the hidden feature of user and the hidden feature of project carries out parallelization, reach the purpose of further lifting matrixes factorization collaborative filtering recommending model extensibility, the present invention carries out according to the following steps:

Step 1, the hidden eigenmatrix of single-row user are or/and the hidden eigenmatrix of project;

For the matrix factorization recommended models of needs structures, with the hidden eigenmatrix P of its user or/and the hidden eigenmatrix Q of project is single-row; By single-row P and Q, so that P, Q only depend on self state value and the initial value of corresponding hidden eigenmatrix before t wheel training beginning in t wheel training process, therefore, for t wheel training process, the training result of every delegation vector of P, Q only depends on initial value and the corresponding training data of the state value of self, corresponding hidden proper vector, and the relation of interdependence between P, Q is eliminated.

Step 2, judge whether hidden eigenmatrix restrains; When the hidden eigenmatrix of user and the hidden eigenmatrix convergence of project, the hidden eigenmatrix of user that the output training is finished is or/and the hidden eigenmatrix of project; When the hidden eigenmatrix of user or/and the hidden eigenmatrix of project when not restraining, execution in step three.

Step 3, the structure based on the hidden proper vector of the user of single-row random Gradient Descent or/and the hidden proper vector training process of project;

According to the hidden eigenmatrix of single-row user and the hidden eigenmatrix of project, hidden eigenmatrix training process to matrix factorization recommended models decomposes, and obtains the hidden eigenmatrix training process of user or/and two training subprocess of the hidden eigenmatrix training process of project; When training moment t, for the capable vectorial P of u among the hidden eigenmatrix P of user _u, its training process based on single-row random Gradient Descent is:

$P_u^t=c^k\·p_u^{t-1}+\mathrm{\η}\underset{k=1}{\overset{M}{\mathrm{\Σ}}}{c}^{M-k}[r_{u,k}-\⟨p_u^{t-1},q_k^{t-1}\⟩]\·q_k^{t-1},$ M is P _uCorresponding training sample number, M is positive integer, and k is the order of the hidden eigenmatrix systematic learning of user training sample, and 1≤k≤M, η are learning rate, and c is learning process stipulations coefficients, c=1-η λ, λ are the learning process standardizing factor, r _{U, k}Be P _uK corresponding training sample; When training moment t, for the capable vector of the i among the hidden eigenmatrix Q of project q _i, its training process based on single-row random Gradient Descent is: $q_i^t=c^k\·q_i^{t-1}+\mathrm{\η}\underset{h=1}{\overset{H}{\mathrm{\Σ}}}{c}^{H-h}[r_{h,i}-\⟨q_i^{t-1},p_h^{t-1}\⟩]\·p_h^{t-1},$ H is q _iCorresponding training sample number, H is positive integer, h is the order of the hidden eigenmatrix systematic learning of project training sample, 1≤h≤H, r _{H, i}Be q _iH corresponding training sample.

The present invention has avoided that the hidden proper vector of each user and the hidden proper vector of project interdepend in the training process of traditional matrix factorization collaborative filtering recommending model, can't carry out the defective of parallelization, can support and analysis means for personalized recommendation research supplies a model.

Better, after described step 3, also comprise the hidden eigenmatrix of parallel training user or/and the step of the hidden eigenmatrix of project, to reach the purpose of further lifting matrixes factorization recommended models extensibility; The hidden eigenmatrix of parallel training user carries out according to the following steps:

A1, according to distributing space Z and training sample set R, successively the capable vector of the m among the hidden eigenmatrix P of user and corresponding training sample thereof are distributed the training node S to correspondence ₁, until

P _zFor distributing training node S ₁Hidden vector set, R _pThe corresponding training sample of the expression hidden proper vector in family;

After A2, the process of distributing are finished, the hidden proper vector of all users is carried out parallel training; The hidden eigenmatrix of parallel training project carries out according to the following steps:

B1, according to distributing space Z and training sample set R, successively the capable vector of the n among the hidden eigenmatrix Q of project and corresponding training sample thereof are distributed the training node S to correspondence ₂, until

Q _zFor distributing to node S ₂Hidden vector set, R _qThe expression project is levied the corresponding training sample of matrix-vector;

After B2, the process of distributing are finished, the hidden proper vector of all projects is carried out parallel training.

The invention has the beneficial effects as follows: the present invention is optimized by the training to the collaborative filtering recommending model, eliminated the relation of interdependence of the hidden eigenmatrix of user and the hidden eigenmatrix of project, improved extensibility, possesses faster speed of convergence, reach the required exercise wheel number of convergence still less, improved the structure speed of recommended models.

Description of drawings

Fig. 1 is the schematic flow sheet of the embodiment of the invention one.

Fig. 2 is the schematic flow sheet of the embodiment of the invention two.

Fig. 3 is the schematic flow sheet of the embodiment of the invention three.

Fig. 4 is the accuracy rate comparison diagram that the present invention and PRMF recommend method.

Fig. 5 is the comparison diagram that the present invention and PRMF recommend the speed-up ratio of method.

Embodiment

The invention will be further described below in conjunction with drawings and Examples:

Embodiment one: as shown in Figure 1, a kind of optimization training method of collaborative filtering recommending model is characterized in that carrying out according to the following steps:

Step 1, the hidden eigenmatrix of single-row user;

For the matrix factorization recommended models of needs structure, that the hidden eigenmatrix P of its user is single-row;

Step 2, judge whether hidden eigenmatrix restrains; When the hidden eigenmatrix convergence of user, the hidden eigenmatrix of user that the output training is finished; When the hidden eigenmatrix of user is not restrained, execution in step three.

Step 3, structure are based on the hidden proper vector training process of the user of single-row random Gradient Descent;

According to the hidden eigenmatrix of single-row user, the hidden eigenmatrix training process of matrix factorization recommended models is decomposed, obtain the training subprocess of the hidden eigenmatrix training process of user; When training moment t, for the capable vectorial P of u among the hidden eigenmatrix P of user _u, its training process based on single-row random Gradient Descent is: $P_u^t=c^k\·p_u^{t-1}+\mathrm{\η}\underset{k=1}{\overset{M}{\mathrm{\Σ}}}{c}^{M-k}[r_{u,k}-\⟨p_u^{t-1},q_k^{t-1}\⟩]\·q_k^{t-1},$ M is P _uCorresponding training sample number, M is positive integer, and k is the order of the hidden eigenmatrix systematic learning of user training sample, and 1≤k≤M, η are learning rate, and c is learning process stipulations coefficients, c=1-η λ, λ are the learning process standardizing factor, r _{U, k}Be P _uK corresponding training sample;

Be r _{U, k}The corresponding hidden proper vector of project is at t-1 state constantly.

Step 4, the hidden eigenmatrix of parallel training user, the hidden eigenmatrix of user that the output training is finished after finishing;

A1, according to distributing space Z and training sample set R, successively the capable vector of the m among the hidden eigenmatrix P of user and corresponding training sample thereof are distributed the training node S to correspondence ₁, until

P _zFor distributing training node S ₁Hidden vector set, R _pThe corresponding training sample of the expression hidden proper vector in family;

After A2, the process of distributing are finished, the hidden proper vector of all users is carried out parallel training.

Embodiment two: as shown in Figure 2, a kind of optimization training method of collaborative filtering recommending model, carry out according to the following steps:

Step 1, the hidden eigenmatrix of single-row project;

For the matrix factorization recommended models of needs structure, that the hidden eigenmatrix Q of its project is single-row;

Step 2, judge whether hidden eigenmatrix restrains; When the hidden eigenmatrix convergence of project, the hidden eigenmatrix of project that the output training is finished; When the hidden eigenmatrix of project is not restrained, execution in step three.

Step 3, structure are based on the hidden proper vector training process of the project of single-row random Gradient Descent;

According to the hidden eigenmatrix of single-row project, the hidden eigenmatrix training process of matrix factorization recommended models is decomposed the training subprocess of the hidden eigenmatrix training process of the project that obtains; When training moment t, for the capable vector of the i among the hidden eigenmatrix Q of project q _i, its training process based on single-row random Gradient Descent is:

H is q _iCorresponding training sample number, H is positive integer, h is the order of the hidden eigenmatrix systematic learning of project training sample, 1≤h≤H, r _{H, i}Be q _iH corresponding training sample;

Be r _{H, i}The corresponding hidden proper vector of user is at t-1 state value constantly.

Step 4, the hidden eigenmatrix of parallel training project, the hidden eigenmatrix of project that the output training is finished after finishing;

B1, according to distributing space Z and training sample set R, successively the capable vector of the n among the hidden eigenmatrix Q of project and corresponding training sample thereof are distributed the training node S to correspondence ₂, until

Q _zFor distributing to node S ₂Hidden vector set, R _qThe expression project is levied the corresponding training sample of matrix-vector;

After B2, the process of distributing are finished, the hidden proper vector of all projects is carried out parallel training.

Embodiment three: as shown in Figure 3, a kind of optimization training method of collaborative filtering recommending model, carry out according to the following steps:

Step 1, the hidden eigenmatrix of single-row user and the hidden eigenmatrix of project;

For the matrix factorization recommended models of needs structure, that the hidden eigenmatrix P of its user and the hidden eigenmatrix Q of project is single-row;

Step 2, judge whether the hidden eigenmatrix of user and the hidden eigenmatrix of project restrain; When the hidden eigenmatrix of user and the hidden eigenmatrix convergence of project, the hidden eigenmatrix of user and the hidden eigenmatrix of project that the output training is finished; When the hidden eigenmatrix of user or the hidden eigenmatrix of project are not restrained, execution in step three.

Step 3, structure are based on the hidden proper vector of the user of single-row random Gradient Descent and the hidden proper vector training process of project;

According to the hidden eigenmatrix of single-row user and the hidden eigenmatrix of project, hidden to matrix factorization recommended models

The eigenmatrix training process decomposes, and obtains two training subprocess of the hidden eigenmatrix training process of user and the hidden eigenmatrix training process of project; When training moment t, for the capable vectorial P of u among the hidden eigenmatrix P of user _u, its training process based on single-row random Gradient Descent is:

$P_u^t=c^k\·p_u^{t-1}+\mathrm{\η}\underset{k=1}{\overset{M}{\mathrm{\Σ}}}{c}^{M-k}[r_{u,k}-\⟨p_u^{t-1},q_k^{t-1}\⟩]\·q_k^{t-1},$ M is P _uCorresponding training sample number, M is positive integer, and k is the order of the hidden eigenmatrix systematic learning of user training sample, and 1≤k≤M, η are learning rate, and c is learning process stipulations coefficients, c=1-η λ, λ are the learning process standardizing factor, r _{U, k}Be P _uK corresponding training sample; When training moment t, for the capable vector of the i among the hidden eigenmatrix Q of project q _i, its training process based on single-row random Gradient Descent is: $q_i^t=c^k\·q_i^{t-1}+\mathrm{\η}\underset{h=1}{\overset{H}{\mathrm{\Σ}}}{c}^{H-h}[r_{h,i}-\⟨q_i^{t-1},p_h^{t-1}\⟩]\·p_h^{t-1},$ H is q _iCorresponding training sample number, H is positive integer, h is the order of the hidden eigenmatrix systematic learning of project training sample, 1≤h≤H, r _{H, i}Be q _iH corresponding training sample;

Step 4, the hidden eigenmatrix of parallel training user and the hidden eigenmatrix of project, the output training is finished after finishing the hidden eigenmatrix of user and the hidden eigenmatrix of project;

The hidden eigenmatrix of parallel training user carries out according to the following steps:

A1, according to distributing space Z and training sample set R, successively the capable vector of the m among the hidden eigenmatrix P of user and corresponding training sample thereof are distributed the training node S to correspondence ₁, until P _zFor distributing training node S ₁Hidden vector set, R _pThe corresponding training sample of the expression hidden proper vector in family;

After A2, the process of distributing are finished, the hidden proper vector of all users is carried out parallel training; The hidden eigenmatrix of parallel training project carries out according to the following steps:

B1, according to distributing space Z and training sample set R, successively the capable vector of the n among the hidden eigenmatrix Q of project and corresponding training sample thereof are distributed the training node S to correspondence ₂, until Q _zFor distributing to node S ₂Hidden vector set, R _qThe expression project is levied the corresponding training sample of matrix-vector;

After B2, the process of distributing are finished, the hidden proper vector of all projects is carried out parallel training.

For correctness and accuracy to method are verified, having moved emulation experiment at the small-size computer cluster verifies, this cluster comprises four nodes altogether, being configured to of each node: INTELi5-760 double-core 2.8GCPU, the internal memory of 8G, in experimental verification, the data set that uses is Movi eLens 1M data set, this data set derives from http://www.grouplens.org/node/12, comprised 6040 users and 3900 projects have been surpassed 1,000,000 score information, its user-project rating matrix consistency is respectively 4.25%, all user's scorings all are distributed in interval [0,5] in, score value is higher, and representative of consumer is stronger to the interest of respective item; RMSE(root mean square error, root-mean-square error are used in experiment) as the evaluation index of recommending accuracy rate, the RMSE value is lower, recommends accuracy rate higher; Experiment uses speed-up ratio as the evaluation index of Algorithm parallelization performance, and speed-up ratio is higher, and the parallelization performance of algorithm is better; The PRMF(parallel Regularized Matrix Factorization that proposes in the algorithm that in the experiment this patent is proposed and the paper " A Paral l el Matrix Factori zat ion basedRecommender by Alternating Stochastic Gradient Decent ", parallel regular matrix decomposes) algorithm compares, this paper and be published in " Engineering Applications of ArtificialIntelligence(engineering and artificial intelligence application) " in 2011.

Wherein lines 1 are for adopting recommendation accuracy rate of the present invention, lines 2 are the recommendation accuracy rate of PRMF algorithm, as seen from Figure 4, the recommendation accuracy rate of the present invention and PRMF method is basic identical, but the method that the present invention proposes possesses faster speed of convergence, reach the required exercise wheel number of convergence still less, as seen from Figure 5, lines 3 are for adopting speed-up ratio of the present invention, lines 4 are the speed-up ratio of PRMF algorithm, along with the increase of concurrent process quantity, the speed-up ratio of the method that the present invention proposes obviously is better than the speed-up ratio of PRMF method, thereby the present invention possesses better parallel performance.

More than describe preferred embodiment of the present invention in detail.Should be appreciated that those of ordinary skill in the art need not creative work and just can design according to the present invention make many modifications and variations.Therefore, all in the art technician all should be in the determined protection domain by claims under this invention's idea on the basis of existing technology by the available technical scheme of logical analysis, reasoning, or a limited experiment.

Claims (2)

1. the optimization training method of a collaborative filtering recommending model is characterized in that carrying out according to the following steps:

Step 1, the hidden eigenmatrix of single-row user and the hidden eigenmatrix of project;

For the matrix factorization recommended models of needs structures, with the hidden eigenmatrix P of its user or/and the hidden eigenmatrix Q of project is single-row;

Step 2, judge whether hidden eigenmatrix restrains; When the hidden eigenmatrix of user or the hidden eigenmatrix of project are not restrained, execution in step three.

Step 3, the structure based on the hidden proper vector of the user of single-row random Gradient Descent or/and the hidden proper vector training process of project;

According to the hidden eigenmatrix of single-row user and the hidden eigenmatrix of project, hidden eigenmatrix training process to matrix factorization recommended models decomposes, and obtains the hidden eigenmatrix training process of user or/and two training subprocess of the hidden eigenmatrix training process of project; When training moment t, for the capable vectorial P of u among the hidden eigenmatrix P of user _u, its training process based on single-row random Gradient Descent is:

$P_u^t=c^k\·p_u^{t-1}+\mathrm{\η}\underset{k=1}{\overset{M}{\mathrm{\Σ}}}{c}^{M-k}[r_{u,k}-\⟨p_u^{t-1},q_k^{t-1}\⟩]\·q_k^{t-1},$ M is P _uCorresponding training sample number, M is positive integer, and k is the order of the hidden eigenmatrix systematic learning of user training sample, and 1≤k≤M, η are learning rate, and c is learning process stipulations coefficients, c=1-η λ, λ are the learning process standardizing factor, r _{U, k}Be P _uK corresponding training sample; When training moment t, for the capable vector of the i among the hidden eigenmatrix Q of project q _i, its training process based on single-row random Gradient Descent is: $q_i^t=c^k\·q_i^{t-1}+\mathrm{\η}\underset{h=1}{\overset{H}{\mathrm{\Σ}}}{c}^{H-h}[r_{h,i}-\⟨q_i^{t-1},p_h^{t-1}\⟩]\·p_h^{t-1},$ H is q _iCorresponding training sample number, H is positive integer, h is the order of the hidden eigenmatrix systematic learning of project training sample, 1≤h≤H, r _{H, i}Be q _iH corresponding training sample;

2. the optimization training method of a kind of collaborative filtering recommending model as claimed in claim 1 is characterized in that: also comprise the hidden eigenmatrix of parallel training user or/and the step of the hidden eigenmatrix of project after described step 3;

The hidden eigenmatrix of parallel training user carries out according to the following steps:

A1, according to distributing space Z and training sample set R, successively the capable vector of the m among the hidden eigenmatrix P of user and corresponding training sample thereof are distributed the training node S to correspondence ₁, until

P _zFor distributing training node S ₁Hidden vector set, R _pThe corresponding training sample of the expression hidden proper vector in family;

After A2, the process of distributing are finished, the hidden proper vector of all users is carried out parallel training; The hidden eigenmatrix of parallel training project carries out according to the following steps:

B1, according to distributing space Z and training sample set R, successively the capable vector of the n among the hidden eigenmatrix Q of project and corresponding training sample thereof are distributed the training node S to correspondence ₂, until

Q _zFor distributing to node S ₂Hidden vector set, R _qThe expression project is levied the corresponding training sample of matrix-vector;

After B2, the process of distributing are finished, the hidden proper vector of all projects is carried out parallel training.

Images