CN102750360B - Mining method of computer data for recommendation systems - Google Patents

Abstract

The invention relates to a mining method of computer data for recommendation systems and belongs to the technical field of computer data processing. The method includes that user preference matrixes and service project preference matrixes are initialized in a main server of a computer, row vectors of preference matrixes input by a user are distributed to a plurality of mappers in the computer, all mappers calculate sub-directions of gradient directions of the user preference matrixes and the service project preference matrixes respectively, calculated results are sent to a simplifier in the computer, the simplifier receives and accumulating the sub-directions of the gradient directions, and the user preference matrixes and the service project preference matrixes are updated according to gradient direction matrixes of the user preference matrixes and the service project preference matrixes. According to the method, the existing probabilistic matrix factorization (PMF) algorithm is modified, large-scale data processing capabilities are improved; and preference matrixes are stored by data storage structures of key-value pairs, so that occupied storage space is small, and reading speed of data are rapid.

Description

A kind of computer data method for digging for commending system

Technical field

The present invention relates to a kind of computer data method for digging for commending system, belong to microcomputer data processing field.

Background technology

Along with the develop rapidly of internet, the continuous expansion of ecommerce scale, commodity number and kind rapid growth, customer need spends the increasing time and could from a large amount of commodity, find and oneself want the commodity bought.This process of browsing a large amount of irrelevant informations and commodity constantly runs off the consumer who is perplexed by problem of information overload.In order to address these problems, personalized recommendation system arises at the historic moment.Personalized recommendation system is based upon on the basis of mass data mining technology, by the data of analysis user, as interest and preference etc., provides decision support and the information service of complete personalization, has improved the flow conversion ratio of e-commerce website.In addition, along with the explosive growth of internet information, information class website user is also faced with the puzzlement of being flooded by magnanimity information.Commending system has promoted user's experience in the use in the fields such as news recommendation, films and television programs recommendation and book recommendation, has increased website viscosity, therefore also has application comparatively widely.

Common commending system algorithm has three kinds of recommendation based on correlation rule (Association Rules), content-based recommendation and collaborative filterings (Collaborative Filtering, CF).Wherein collaborative filtering is most widely used in commending system.But still there are many problems urgently to be resolved hurrily.Generally speaking, Deta sparseness (Sparsity) problem and system ductility (Scalability) problem are the common challenges that all collaborative filterings face.

In prior art, " R.Salakhutdinov and A.Mnih; " Probabilistic Matrix Factorization "; Advances in Neural Information Processing Systems 20 (NIPS ' 07); pp.1257 – 1264; 2008 ", a kind of probability matrix decomposition algorithm excavating for computer data is disclosed (hereinafter to be referred as PMF, Probabilistic Matrix Factorization) this is the matrix decomposition algorithm based on bayesian theory proposing in recent years.PMF model can be used for based on scoring (ratings) collaborative filtering, not only there is good system extension (increasing linear than relation with user), and in sparse data exhibits excellent.As a kind of linear factor model, in PMF model, user's preference can be expressed as the vector that the linear combination of the preference factor to service item by user forms.User can be expressed as the user preference matrix U of a N × D to the preference matrix R of N × M ^twith the product of the commodity factor matrix V of a D × M, i.e. R=U ^tv.The best D dimension of training such equivalent model to find objective matrix R under the condition at given loss function is similar to.Secondly, PMF is the matrix decomposition algorithm based on bayesian theory.By giving rational prior probability distribution to U, V matrix, and according to R ≈ U ^tv relation obtains the likelihood function of R matrix, can calculating parameter matrix U, the posterior probability of V distributes, and finally uses the maximum a posteriori probability estimation technique to try to achieve U, V matrix.

Illustrate: suppose to have N user and M film, user is the integer from 1 to K to the score value of film.R _ijrepresent the scoring of user i to film j, U ∈ R ^{d × N}with V ∈ R ^{d × M}it is the potential eigenmatrix of user and film.Defining observable all score data R with respect to the likelihood function of potential feature U, V is:

$p=\left(R\right|U,V,{\mathrm{\σ}}^2)=\underset{i=1}{\overset{N}{\mathrm{\Π}}}\underset{j=1}{\overset{M}{\mathrm{\Π}}}{\left[N\left(R_{\mathrm{ij}}\right|U_i^T{V}_j,{\mathrm{\σ}}^2)\right]}^{I_{\mathrm{ij}}}$

Formula 1

Wherein N (x| μ, σ ²) expression average is that μ, variance are σ ²the probability density function of Gaussian distribution.I _ijto represent the whether indicative function to film j scoring of user i.It is 0 Gaussian distribution that the prior distribution of supposing user characteristics matrix U and movie features matrix V meets average:

$p\left(U\right|{\mathrm{\σ}}_U^2)=\underset{i=1}{\overset{N}{\mathrm{\Π}}}N\left(U_i\right|0,{\mathrm{\σ}}_U^{2}I),p\left(V\right|{\mathrm{\σ}}_V^2)=\underset{j=1}{\overset{M}{\mathrm{\Σ}}}N\left(V_j\right|0,{\mathrm{\σ}}_V^{2}I)$

Formula 2

The logarithm expression formula that can obtain the posterior probability distribution of U, V matrix is:

$\ln p(U,V|R,{\mathrm{\σ}}^2,{\mathrm{\σ}}_V^2,{\mathrm{\σ}}_U^2)=-\frac{1}{2{\mathrm{\σ}}^2}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{I}_{\mathrm{ij}}{(R_{\mathrm{ij}}-U_i^T{V}_j)}^2-\frac{1}{2{{\mathrm{\σ}}_U}^2}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{U}_i^T{U}_i-\frac{1}{2{{\mathrm{\σ}}_V}^2}\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{V}_j^T{V}_j+C$

Formula 3

Maximize posterior probability and be equivalent to the optimization problem with minor function:

$E=\frac{1}{2}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{I}_{\mathrm{ij}}{(R_{\mathrm{ij}}-U_i^T{V}_j)}^2+\frac{{\mathrm{\λ}}_U}{2}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\left|\right|U_i\left|\right|}_{\mathrm{Fro}}^2+\frac{{\mathrm{\λ}}_V}{2}\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{\left|\right|V_j\left|\right|}_{\mathrm{Fro}}^2$

Formula 4

λ herein _u=σ ²/ σ _u ², λ _v=σ ²/ σ _v ², depend on that user inputs.‖ ‖ _frorepresent Fu Luobin Nice norm.

Like this, the derivation of PMF model parameter is just reduced to common Unconstrained Optimization Problem, by gradient descent method, can obtain easily U, V matrix.

Along with using the continuous expansion of website scale of commending system, the requirement of the storage capacity of the PMF algorithm under stand-alone environment to server and computing power is more and more higher, cannot meet real needs.

PMF algorithm under stand-alone environment need to calculate whole rating matrix R reading in internal memory, and while carrying out iterative with gradient descent method, whether U, the V matrix of trying to achieve in judgement have met error requirements, need to calculate the product matrix of U and V.This explanation PMF algorithm at least should put down two R matrixes to the requirement of internal memory.This is feasible in the time that data scale is little.And in real world applications scene, the user characteristics matrix of large scale business website and dimension N, the M of product features matrix often 1,000,000, even ten million magnitude.Now, the internal memory of common server can not meet the requirement to internal memory of PMF algorithm.

In addition, PMF is a kind of iterative algorithm, and each iteration all needs to carry out matrix multiplication and check whether reach iteration border, so the time complexity calculating is very high.If the iterations of Gradient Descent is k time, the time complexity of PMF algorithm is O (kNMD).In the time that user, product features matrix become larger, the time that the PMF algorithm under stand-alone environment needs is oversize.

Summary of the invention

The object of the invention is to propose a kind of computer data method for digging for commending system, consider being on a grand scale of preference matrix R, R matrix is positioned over to external memory, and consider when R matrix size is very large, normally Sparse Array of R matrix, by after the R matrix disposal of input, adopts more compact data structure to store R matrix, to shorten computing time, increase the treatment scale of data.

The computer data method for digging for commending system that the present invention proposes, comprises the following steps:

(1) the preference matrix R of a N × M of setting, wherein N is the line number of preference matrix R, and N equals user's number, and M is the columns of preference matrix R, and M equals the project number for user's service;

(2) to computer input file, convert input file to sequential file in mapping abbreviation model, make a row vector of each the behavior preference matrix R in sequential file, the data structure of every a line of preference matrix R is: row vector subscript and key-value pair array composition, and wherein key-value pair array comprises service item numbering and the preference of user to this service item;

(3) preference matrix R is expressed as to R=U ^tv, wherein U ^tfor the user preference transpose of a matrix of N × D, N equals user's number, and D is user's service item preference factor number, the service item preference matrix that V is D × M, and M is service item number;

(4) in the master server of computing machine, generate user preference matrix U and service item preference matrix V, the wherein behavior Customs Assigned Number of user preference matrix U, classify the user preference factor as, when initialization, the user preference factor is any real number, the behavior service item numbering of service item preference matrix V, classify the service item preference factor as, and the service item preference factor is any real number while establishing initialization;

(5) row vector of above-mentioned preference matrix R is distributed to the multiple mappers in computing machine, each mapper is according to the row vector of the preference matrix R reading, respectively according to formula:

$\▿U_{\mathrm{ik}}={\mathrm{\λ}}_U{U}_{\mathrm{ik}}+\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{I}_{\mathrm{ij}}{V}_j(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ The gradient direction ▽ U of each element in compute user preferences matrix U _ik,

According to formula: $\mathrm{\Δ}(\▿V_{\mathrm{ik}})=I_{\mathrm{ij}}{U}_i(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ Sub-direction Δ (the ▽ V of the gradient direction of each element in calculation services project preference matrix V _ik),

Wherein,

represent i row vector of the transposition of user preference matrix U, V _jrepresent j the row vector of service item preference matrix V, λ _uthe user preference extent index that user specifies, λ _ufor arithmetic number, U _ikfor the element that i is capable, k is listed as of user preference matrix U, I is indicator function matrix, if I _ijequal 0, represent that user i does not produce preference to service item j, if I _ijequal 1, represent that user i produces preference to service item j, g is Rogers's number of writing, and g ' is the single order derived function of g function:

$g\left(x\right)=\frac{1}{1+e^{-x}}$

Each mapper is by gradient direction ▽ U _ikwith sub-direction Δ (the ▽ V of gradient direction _ik) send to respectively the abbreviation device in computing machine;

(6) abbreviation device is according to sub-direction Δ (the ▽ V of the gradient direction of each element in the service item preference matrix V receiving _ik) add up, obtain the gradient direction ▽ V of each element of matrix V _ik,

wherein λ _vthe service item preference extent index that user specifies, λ _vit is arithmetic number;

(7) abbreviation device builds the gradient direction matrix ▽ U of a user preference matrix U, and in gradient direction matrix ▽ U, i value capable, k row is the ▽ U that step (5) calculates _ik, building the gradient direction matrix ▽ V of a service item preference matrix V, in gradient direction matrix ▽ V, i value capable, k row is the ▽ V that step (5) calculates _ik;

And according to the gradient direction matrix ▽ V of the gradient direction matrix ▽ U of user preference matrix U and service item preference matrix V, user preference matrix U and service item preference matrix V are upgraded, make:

$\begin{array}{c}U=U-\▿U\\ V=V-\▿V\end{array},$ Complete iteration one time;

(8) user sets a maximum iteration time, if iterations is more than or equal to maximum iteration time, obtains user preference matrix U and service item preference matrix V, finishes to calculate; If iterations is less than maximum iteration time, repeating step (3)～step (8).

The computer data method for digging for commending system that the present invention proposes, its advantage is:

1, the inventive method is improved existing PMF algorithm, has improved large-scale data processing power.The inventive method is stored in preference matrix R in the file of external memory, based on mapping abbreviation (MapReduce) framework, data are carried out to distributed concurrent, requiring of computing power to single computing machine in computing system and storage capacity is low, and the probability that effectively adapts to extensive matrix decomposes.

2, in the inventive method, adopt the data store organisation of key-value pair to store preference matrix R, the storage area that R is taken is less, and data reading speed is faster.

3, the inventive method adopts concurrent method to the gradient direction of user preference matrix U, service item preference matrix V, in the iteration each time of gradient descent method, each row vector of R only need to be read once, can calculate concomitantly the gradient direction of U, V, so the reading times to R matrix is few, data mining speed is fast.

Embodiment

The computer data method for digging for commending system that the present invention proposes, comprises the following steps:

(1) the preference matrix R of a N × M of setting, wherein N is the line number of preference matrix R, and N equals user's number, and M is the columns of preference matrix R, and M equals the project number for user's service;

(2) to computer input file, convert input file to sequential file in mapping abbreviation model, make a row vector of each the behavior preference matrix R in sequential file, the data structure of every a line of preference matrix R is: row vector subscript and key-value pair array composition, and wherein key-value pair array comprises service item numbering and the preference of user to this service item;

The data structure of every a line of preference matrix R is as shown in the table, and wherein user i produces preference to m service item altogether:

In upper table, to the capable R of i of preference matrix R _i, first storage line vector subscript i.Then for each service item j of user i preference, the row subscript of service item in preference matrix R is the key K in key-value pair, and user i is corresponding value V to the preference of service item j.

(3) preference matrix R is expressed as to R=U ^tv, wherein U ^tfor the user preference transpose of a matrix of N × D, N equals user's number, and D is user's service item preference factor number, the service item preference matrix that V is D × M, and M is service item number;

(4) in the master server of computing machine, generate user preference matrix U and service item preference matrix V, the wherein behavior Customs Assigned Number of user preference matrix U, classify the user preference factor as, when initialization, the user preference factor is any real number, the behavior service item numbering of service item preference matrix V, classify the service item preference factor as, and the service item preference factor is any real number while establishing initialization;

(5) row vector of above-mentioned preference matrix R is distributed to the multiple mappers in computing machine, each mapper is according to the row vector of the preference matrix R reading, respectively according to formula:

$\▿U_{\mathrm{ik}}={\mathrm{\λ}}_U{U}_{\mathrm{ik}}+\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{I}_{\mathrm{ij}}{V}_j(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ The gradient direction ▽ U of each element in compute user preferences matrix U _ik,

According to formula: $\mathrm{\Δ}(\▿V_{\mathrm{ik}})=I_{\mathrm{ij}}{U}_i(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ Sub-direction Δ (the ▽ V of the gradient direction of each element in calculation services project preference matrix V _ik),

Wherein,

represent i row vector of the transposition of user preference matrix U, V _jrepresent j the row vector of service item preference matrix V, λ _uthe user preference extent index that user specifies, λ _ufor arithmetic number, U _ikfor the element that i is capable, k is listed as of user preference matrix U, I is indicator function matrix, if I _ijequal 0, represent that user i does not produce preference to service item j, if I _ijequal 1, represent that user i produces preference to service item j, g is Rogers's number of writing, and g ' is the single order derived function of g function:

$g\left(x\right)=\frac{1}{1+e^{-x}}$

Each mapper is by gradient direction ▽ U _ikwith sub-direction Δ (the ▽ V of gradient direction _ik) send to respectively the abbreviation device in computing machine; (6) abbreviation device is according to sub-direction Δ (the ▽ V of the gradient direction of each element in the service item preference matrix V receiving _ik) add up, obtain the gradient direction ▽ V of each element of matrix V _ik,

wherein λ _vthe service item preference extent index that user specifies, λ _vit is arithmetic number;

(7) abbreviation device builds the gradient direction matrix ▽ U of a user preference matrix U, and in gradient direction matrix ▽ U, i value capable, k row is the ▽ U that step (5) calculates _ik, building the gradient direction matrix ▽ V of a service item preference matrix V, in gradient direction matrix ▽ V, i value capable, k row is the ▽ V that step (5) calculates _ik;

And according to the gradient direction matrix ▽ V of the gradient direction matrix ▽ U of user preference matrix U and service item preference matrix V, user preference matrix U and service item preference matrix V are upgraded, make:

$\begin{array}{c}U=U-\▿U\\ V=V-\▿V\end{array},$ Complete iteration one time;

(8) user sets a maximum iteration time, if iterations is more than or equal to maximum iteration time, obtains user preference matrix U and service item preference matrix V, finishes to calculate; If iterations is less than maximum iteration time, repeating step (3)～step (8).

Claims (1)

1. for a computer data method for digging for commending system, it is characterized in that the method comprises the following steps:

(1) the preference matrix R of a N × M of setting, wherein N is the line number of preference matrix R, and N equals user's number, and M is the columns of preference matrix R, and M equals the project number for user's service;

(2) to computer input file, convert input file to sequential file in mapping abbreviation model, make a row vector of each the behavior preference matrix R in sequential file, the data structure of every a line of preference matrix R is: row vector subscript and key-value pair array composition, and wherein key-value pair array comprises service item numbering and the preference of user to this service item;

(3) preference matrix R is expressed as to R=U ^tv, wherein U ^tfor the user preference transpose of a matrix of N × D, N equals user's number, and D is user's service item preference factor number, the service item preference matrix that V is D × M, and M is service item number;

(4) in the master server of computing machine, generate user preference matrix U and service item preference matrix V, the wherein behavior Customs Assigned Number of user preference matrix U, classify the user preference factor as, when initialization, the user preference factor is any real number, the behavior service item numbering of service item preference matrix V, classify the service item preference factor as, and the service item preference factor is any real number while establishing initialization;

(5) row vector of above-mentioned preference matrix R is distributed to the multiple mappers in computing machine, each mapper is according to the row vector of the preference matrix R reading, respectively according to formula:

${\▿U}_{\mathrm{ik}}={\mathrm{\λ}}_U{U}_{\mathrm{ik}}+\underset{j=1}{\overset{M}{\mathrm{\Σ}}}{I}_{\mathrm{ij}}{V}_j(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ The gradient direction ▽ U of each element in compute user preferences matrix U _ik,

According to formula: $\mathrm{\Δ}\left({\▿V}_{\mathrm{ik}}\right)=I_{\mathrm{ij}}{U}_i(R_{\mathrm{ij}}-g\left(U_i^T{V}_j\right))g^{\′}\left(U_i^T{V}_j\right),$ Sub-direction Δ (the ▽ V of the gradient direction of each element in calculation services project preference matrix V _ik),

Wherein,

represent i row vector of the transposition of user preference matrix U, V _jrepresent j the row vector of service item preference matrix V, λ _uthe user preference extent index that user specifies, λ _ufor arithmetic number, U _ikfor the element that i is capable, k is listed as of user preference matrix U, I is indicator function matrix, if I _ijequal 0, represent that user i does not produce preference to service item j, if I _ijequal 1, represent that user i produces preference to service item j, g is Rogers's number of writing, and g ' is the single order derived function of g function:

$g\left(x\right)=\frac{1}{1+e^{-x}};$

Each mapper is by gradient direction ▽ U _ikwith sub-direction Δ (the ▽ V of gradient direction _ik) send to respectively the abbreviation device in computing machine;

(6) abbreviation device is according to sub-direction Δ (the ▽ V of the gradient direction of each element in the service item preference matrix V receiving _ik) add up, obtain the gradient direction ▽ V of each element of matrix V _ik, wherein λ _vthe service item preference extent index that user specifies, λ _vit is arithmetic number;

(7) abbreviation device builds the gradient direction matrix ▽ U of a user preference matrix U, and in gradient direction matrix ▽ U, i value capable, k row is the ▽ U that step (5) calculates _ik, building the gradient direction matrix ▽ V of a service item preference matrix V, in gradient direction matrix ▽ V, i value capable, k row is the ▽ V that step (5) calculates _ik;

And according to the gradient direction matrix ▽ V of the gradient direction matrix ▽ U of user preference matrix U and service item preference matrix V, user preference matrix U and service item preference matrix V are upgraded, make:

$\begin{array}{c}U=U-\▿U\\ V=V-\▿V\end{array},$ Complete iteration one time;

(8) user sets a maximum iteration time, if iterations is more than or equal to maximum iteration time, obtains user preference matrix U and service item preference matrix V, finishes to calculate; If iterations is less than maximum iteration time, repeating step (3)～step (8).