CN103049489B - For the treatment of the method and system of collaborative filtering distance - Google Patents
For the treatment of the method and system of collaborative filtering distance Download PDFInfo
- Publication number
- CN103049489B CN103049489B CN201210518168.2A CN201210518168A CN103049489B CN 103049489 B CN103049489 B CN 103049489B CN 201210518168 A CN201210518168 A CN 201210518168A CN 103049489 B CN103049489 B CN 103049489B
- Authority
- CN
- China
- Prior art keywords
- calculation server
- calculating
- current
- round
- matrix data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 102
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004364 calculation method Methods 0.000 claims abstract description 538
- 239000011159 matrix material Substances 0.000 claims abstract description 176
- 230000006854 communication Effects 0.000 claims abstract description 108
- 238000004891 communication Methods 0.000 claims abstract description 100
- 238000006243 chemical reaction Methods 0.000 claims abstract description 8
- 230000005540 biological transmission Effects 0.000 claims description 23
- 230000006870 function Effects 0.000 claims description 17
- 230000000052 comparative effect Effects 0.000 claims description 16
- 230000001143 conditioned effect Effects 0.000 claims description 16
- 238000005516 engineering process Methods 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000011800 void material Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000001303 quality assessment method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Abstract
The invention discloses a kind of method and system for the treatment of collaborative filtering distance of the present invention, relate to field of computer technology.Described system comprises: N number of calculation server; Described each calculation server participating in calculating comprises: digital independent memory module; Local data computing module; Communication cycle computing module; Described communication cycle computing module comprises object calculation server and confirms module; Matrix data sending/receiving module; Cooperated computing module; Sorting module, is suitable for all collaborative filtering submatrixs that each calculation server participating in calculating calculates to carry out unifying conversion, obtains complete collaborative filtering matrix.The invention solves and can only calculate by single calculation server the problem obtaining collaborative filtering distance matrix, achieve under internet environment, the counting yield of the collaborative filtering distance matrix for mass data can be improved, reduce the beneficial effect of computing time.
Description
Technical field
The present invention relates to field of computer technology, be specifically related to a kind of method and system for the treatment of collaborative filtering distance.
Background technology
Information resources on internet exponentially expand and bring so-called " information overload " and " information is isotropic " problem, and namely people are difficult to find oneself interested information, even if having found, are also often mixed with a lot " noise ".Therefore the technology such as the information retrieval of Internet, information filtering and collaborative filtering have been there is.But information retrieval does not have intelligent, the interest of user can not be learnt, especially to the user with particular professional interest, input identical keyword and can only obtain identical result for retrieval.Information filtering can not distinguish quality to the filter result of same subject, and along with the sharp increase of information resources, more effective filtration needs the quality assessment information in conjunction with people.Based on this kind of demand, occurred commending system, commending system is a kind of intelligent proxy system proposed for solving problem of information overload, can automatically recommend out the resource meeting its interest preference or demand from bulk information to user.Along with the universal of internet and develop rapidly, commending system has been widely used in various field, and especially in e-commerce field, commending system obtains increasing investigation and application.At present, nearly all electronic business web site all in various degree employ various forms of commending system, bookstore etc. on such as Amazon, CDNOW, eBay and Dangdang.com.Wherein, collaborative filtering obtains larger success in the application of current commending system.
Collaborative filtering is a class proposed algorithm.Collaborative filtering considers the evaluation information of user.Collaborative filtering analyzes user interest, and in customer group, find similar (interest) user of designated user, comprehensively these similar users are to the evaluation of a certain information, forms system to the prediction of this designated user to the fancy grade of this information.Mainly be divided into the collaborative filtering (Userbased) based on user and (Itembased) two kinds based on commodity.Such as based on the collaborative filtering of user, its basic thought is: if will be the commodity that a user recommends it not have, first calculate the similarity degree of the hobby of other user and this user, the commodity then finding this user not have from some users the most similar to its hobby are recommended.Collaborative filtering principle based on commodity is similar.Therefore in algorithm, a very important step calculates the similarity degree (being also collaborative filtering distance) between all users.
Under internet environment, user and commodity are all mass datas, simultaneously because actual effect has regular hour requirement to algorithm, and be carry out on a calculation server substantially for the calculating of collaborative filtering distance in prior art, and when meeting with super amount user, such as when 1,000,000 order of magnitude, Single-Server is adopted to calculate then timeliness too slow, and higher for the requirement of hardware.
Summary of the invention
In view of the above problems, the present invention is proposed to provide a kind of overcoming the problems referred to above or a kind of system for the treatment of collaborative filtering distance solved the problem at least in part and a kind of method for the treatment of collaborative filtering distance accordingly.
According to one aspect of the present invention, provide a kind of method for the treatment of collaborative filtering distance, comprising:
For at least two calculation servers, the calculation server that each participation calculates reads and stores corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Each participation calculates the partitioned matrix data that calculation server calculates this locality, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
The communication cycle that each calculation server participating in calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data; The communication cycle that described each calculation server participating in calculating carries out between each calculation server calculates, and obtains collaborative filtering distance submatrix and specifically comprises:
For each calculation server participating in calculating, confirm the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Each calculation server participating in calculating sends local partitioned matrix data to object calculation server, and receives the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received;
All collaborative filtering submatrixs that each calculation server participating in calculating calculates are carried out unifying conversion, obtains complete collaborative filtering matrix.
Optionally, described for each calculation server participating in calculating, confirm that the object calculation server that current calculation server is corresponding comprises:
Described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding.
Optionally, described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm that the object calculation server that current calculation server is corresponding comprises:
By carrying out the round p of communication cycle calculating between current each calculation server, compare with the half N/2 of total number N of calculation server;
If comparative result meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
If comparative result does not meet pre-conditioned, then terminate the calculating obtaining collaborative filtering distance submatrix.
Optionally, describedly will carry out the round p of communication cycle calculating between current each calculation server, also comprise after comparing with the half N/2 of total number N of calculation server:
Judge whether total number N of calculation server is even number;
If total number N of calculation server is even number, then judge whether present communications round p is that last takes turns communication;
If current round p is last take turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2;
If mark i is less than N/2, then identify the calculation server that i is less than N/2, enter the mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
If mark i is not less than N/2, then enters the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server;
If current round p is last take turns communication, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
If total number N of calculation server is odd number, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server.
Optionally, when described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
Optionally, described each calculation server participating in calculating communication cycle carried out between each calculation server calculates and comprises:
The communication cycle that each calculation server participating in calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
According to another aspect of the present invention, provide a kind of system for the treatment of collaborative filtering distance, comprising:
N number of calculation server;
Described each calculation server participating in calculating comprises:
Digital independent memory module, is suitable for each calculation server participating in calculating and reads and store corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Local data computing module, is suitable for each calculation server participating in calculating and calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Communication cycle computing module, the communication cycle that the calculation server being suitable for each participation calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Described communication cycle computing module comprises:
Object calculation server confirms module, is suitable for, for each calculation server participating in calculating, confirming the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Matrix data sending/receiving module, is suitable for each calculation server participating in calculating and sends local partitioned matrix data to object calculation server, and receive the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Cooperated computing module, is suitable for each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received;
Sorting module, is suitable for all collaborative filtering submatrixs that each calculation server participating in calculating calculates to carry out unifying conversion, obtains complete collaborative filtering matrix.
Optionally, described object calculation server confirms that module comprises:
First object calculation server confirms module, be suitable for described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding.
Optionally, described first object calculation server confirms that module comprises:
Stop judge module, being suitable for the round p by carrying out communication cycle calculating between current each calculation server, comparing with the half N/2 of total number N of calculation server;
Second destination server confirms submodule, if the comparative result being suitable for judging meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
First stopping modular, if the comparative result being suitable for judging does not meet pre-conditioned, then terminates the calculating obtaining collaborative filtering distance submatrix.
Optionally, also comprise after described stopping judge module:
Second judge module, is suitable for judging whether total number N of calculation server is even number;
3rd judge module, if the total number N being suitable for calculation server is even number, then judges whether present communications round p is that last takes turns communication;
4th judge module, if being suitable for current round p is that last takes turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2;
3rd destination server confirms submodule, if be suitable for mark i to be less than N/2, then identify the calculation server that i is less than N/2, enter the mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
Proceed to module, if be suitable for mark i to be not less than N/2, then enter the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server;
3rd destination server confirms module, if the total number N being suitable for calculation server is odd number, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server.
Optionally, when described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
Optionally, described matrix data sending/receiving module comprises:
First matrix data sending/receiving module, the communication cycle that the calculation server being suitable for each participation calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
According to each collaborative filtering distance submatrix calculating former collaborative filtering distance matrix that a kind of method for the treatment of collaborative filtering distance of the present invention can be synchronous in individual calculation server, solve thus and can only calculate by single calculation server the problem obtaining collaborative filtering distance matrix, achieve under internet environment, the counting yield of the collaborative filtering distance matrix for mass data can be improved, reduce the beneficial effect of computing time.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to technological means of the present invention can be better understood, and can be implemented according to the content of instructions, and can become apparent, below especially exemplified by the specific embodiment of the present invention to allow above and other objects of the present invention, feature and advantage.
Accompanying drawing explanation
By reading hereafter detailed description of the preferred embodiment, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of preferred implementation, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 shows a kind of according to an embodiment of the invention schematic flow sheet of the embodiment of the method one for the treatment of collaborative filtering distance;
Fig. 2 shows a kind of according to an embodiment of the invention schematic flow sheet of the embodiment of the method two for the treatment of collaborative filtering distance;
The communication example of 4 calculation servers has according to an embodiment of the invention been shown in Fig. 3;
The communication example of 5 calculation servers has according to an embodiment of the invention been shown in Fig. 4;
Fig. 5 shows a kind of according to an embodiment of the invention structural representation of the system embodiment one for the treatment of collaborative filtering distance; And
Fig. 6 shows a kind of according to an embodiment of the invention structural representation of the system embodiment two for the treatment of collaborative filtering distance.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
With reference to Fig. 1, it illustrates the schematic flow sheet of a kind of embodiment of the method one for the treatment of collaborative filtering distance of the present invention, specifically can comprise:
Step 110, at least two calculation servers, the calculation server that each participation calculates reads and stores corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
For convenience of describing, at this first for the collaborative filtering of Userbase, introduce User (user) filters distance matrix principle to the coordination of Item (column, column can comprise commodity classification etc. usually):
Introduce User and distance matrix is filtered to the coordination of Item, namely User is calculated to Item similar matrix, that is to say and refer to according to the scoring matrix of User to Item, calculate the similarity between every two User, so obtain between User similar matrix (namely coordinate filter distance matrix).Therefore, obtain final similar matrix, once Sim computing (similarity computing) will be carried out with other row in matrix in each provisional capital.Detailed process, as follows
wherein
Wherein,
in every behavior user m to the marking of project n, s
ijfor the similarity between user i and user j,
be the row vector of the i-th row, Sim function can have multiple choices, such as COS distance, Pearson (pearson) related coefficient, Euclidean distance etc.
In Large-scale parallel computing, Data distribution8, on multiple stage calculation server, therefore will consider the problem of local data and external data in computation process.If data divide according to row, namely each calculation server participating in calculating preserves the data of part row, then the corresponding relation of calculation server and data is as follows:
Node
0:
Node
1:
……
Node
n:
So in the embodiment of the present invention, for multiple calculation server, the user-project scoring matrix that first can will store
piecemeal distributes to each calculation server, in embodiments of the present invention, and general distributed by row.If the matrix certainly stored is
transposed matrix, so also can by row distribute.
After distributing the corresponding relation of the matrix of initial reading corresponding to aforementioned scoring matrix for each calculation server participating in calculating, after then starting to start the calculating of collaborative filtering distance, each calculation server participating in calculating reads and stores corresponding partitioned matrix data, and wherein said partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server.
Wherein, in the embodiment of the present invention, original matrix data (i.e. whole marking data) are stored in private server, and so each calculation server then reads corresponding matrix data from this private server.
In embodiments of the present invention, for convenience of description, follow-up all for the angle of Userbase, to the scoring matrix of User to Item
for example is described similar matrix (collaborative filtering distance matrix).For Itembase angle, only need by
transposition, principle is similar.
What may participate in parallel computation in embodiments of the present invention comprises N number of calculation server, and N is more than or equal to 2.
Step 120, each calculation server participating in calculating calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Each calculation server participating in calculating then calculates the local initial partitioned matrix data read, and obtains the collaborative filtering distance submatrix of local partitioned matrix data.The matrix that such as aforementioned calculation server 0 reads is
so calculation server 0 calculates the similarity (i.e. collaborative filtering distance) in this partitioned matrix between every two row, so namely obtains
in the similarity matrix of a part, the also collaborative filtering submatrix of i.e. local data.
Step 130, the communication cycle that each calculation server participating in calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data;
In embodiments of the present invention, in order to ensure for making each calculation server can calculate all collaborative filtering submatrixs together, namely the calculating that in former scoring matrix, every two row all can not leak, to obtain the matrix of complete similarity, each calculation server is so needed to carry out Memory communication in computation process.Each calculation server participating in calculating is calculating the local partitioned matrix data stored, after obtaining the collaborative filtering distance submatrix of local partitioned matrix data, the communication cycle that each calculation server participating in calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data.
In embodiments of the present invention, each calculation server of the present invention adopts MPI (MessagePassingInterface, message passing interface; A kind of program message passing interface, provides the multilingual function library realizing one series interfaces simultaneously) carry out between calculation server communication cycle.
Optionally, described each calculation server participating in calculating communication cycle carried out between each calculation server calculates and comprises:
The communication cycle that each calculation server participating in calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
For MPI_send, its function roughly example is as follows:
intMPI_Send(void*buf,intcount,MPI_Datatypedatatype,intdest,inttag,MPI_Commcomm)；
For MPI_recv, its function roughly example is as follows:
intMPI_Recv(void*buf,intcount,MPI_Datatypedatatype,intsource,inttag,MPI_Commcomm,MPI_Status*status)；
The communication cycle that described each calculation server participating in calculating carries out between each calculation server calculates, and obtains collaborative filtering distance submatrix and specifically comprises:
Step 131, for each calculation server participating in calculating, confirms the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Each calculation server participating in calculating then needs the destination server confirming current server, has sent local partitioned matrix data to destination server.
Optionally, in the embodiment of the present invention, described for each calculation server participating in calculating, confirm that the object calculation server that current calculation server is corresponding comprises:
Step S131, for each calculation server participating in calculating, calculates round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirms the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
In embodiments of the present invention, in order to manage each calculation server participating in calculating, the mark of calculation server can be set for each calculation server, such as numeral 0,1 ... Deng.Each calculation server participating in calculating can be notified simultaneously, total number N of calculation server in whole parallel computation network, each server also can record the round of the integrated communication in whole parallel computation network between calculation server, so for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission.
Optionally, described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm that the object calculation server that current calculation server is corresponding comprises:
Step S11, by carrying out the round p of communication cycle calculating between current each calculation server, compares with the half N/2 of total number N of calculation server;
Such as when described round counts from 0, judge the round p carrying out communication cycle calculating between current each calculation server, whether be less than the half N/2 of total number N of calculation server.
Such as when described round counts from 1, judge the round p carrying out communication cycle calculating between current each calculation server, whether be less than or equal to the half N/2 of total number N of calculation server.
In embodiments of the present invention, in order to avoid each calculation server double counting, be provided with the highest round that each calculation server carries out communication cycle calculating, the highest optimum round is N/2, namely communicating, round is the highest can only arrive N/2, and certainly the highest round also can be greater than N/2.
Step S12, if comparative result meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
For step S12, such as when described round counts from 0, if the round p carrying out partitioned matrix data communication between current calculation server is less than N/2, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server.
When described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
Step S012, by the mark i of current calculation server and round p, substitutes into formula (i+p+1) %N and calculates result u;
Conveniently calculation server calculates, and the mark i of calculation server can be set to the integer from 0, certainly other forms also can, need be only the integer other forms of mark is converted to from 0 in calculating.
Such as i=0, round p=0, calculation server adds up to 4, so (i+p+1) %N=1, and namely u is 1,1 corresponding calculation server 1, and so the object calculation server of calculation server 0 is calculation server 1.
For step S12, such as when described round counts from 1, if the round p carrying out partitioned matrix data communication between current calculation server is less than or equal to N/2, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server.
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
Step S013, by the mark i of current calculation server and round p, substitutes into formula (i+p) %N and calculates result u.
Such as i=0, round p=1, calculation server adds up to 4, so (i+p) %N=1, and namely u is 1,1 corresponding calculation server 1, and so the object calculation server of calculation server 0 is calculation server 1.
Step S13, if comparative result does not meet pre-conditioned, then terminates the calculating obtaining collaborative filtering distance submatrix.
Such as, when described round counts from 0, if the round p carrying out partitioned matrix data communication between current calculation server is more than or equal to N/2, then terminate the calculating obtaining collaborative filtering distance submatrix.
Such as, when described round counts from 1, if the round p carrying out partitioned matrix data communication between current calculation server is greater than N/2, then terminate the calculating obtaining collaborative filtering distance submatrix.
Step 132, each calculation server participating in calculating sends local partitioned matrix data to object calculation server, and receives the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
After each calculation server participating in calculating confirms its object calculation server, its local partitioned matrix data stored are sent to destination server, and accept the partitioned matrix of other calculation servers transmission.Such as aforementioned Node
0:
Node
1:
……
Node
n:
Node
0stored
send to Node
1, accept Node
nsend
then other calculation servers Node
isimilar.
Step 133, each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received.
Calculation server Node as previously mentioned
0calculate
in each row vector with
in each row vector between similarity, obtain
with
between similar matrix (i.e. collaborative filtering submatrix), other calculation servers are similar.
In embodiments of the present invention, the partitioned matrix data that each calculation server participating in calculating sends over for other calculation servers, after calculating, the partitioned matrix data release that this other calculation server can be sent, will not store.
Optionally, also comprise:
All collaborative filtering submatrixs that each calculation server participating in calculating calculates are carried out unifying conversion, obtain complete collaborative filtering matrix by step 140.
Each calculation server participating in calculating, at the sub-square of collaborative filtering calculating a part, can send it to aforementioned private server, then carries out unifying to arrange storing in private server.
With reference to Fig. 2, it illustrates the schematic flow sheet of a kind of embodiment of the method two for the treatment of collaborative filtering distance of the present invention, specifically can comprise:
Step 210, for N number of calculation server, the calculation server that each participation calculates reads and stores corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Step 220, each calculation server participating in calculating calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Step 210,220 and Fig. 1 to describe 110,120 principles substantially similar, be not described in detail in this.
Step 230, each calculation server participating in calculating, by carrying out the round p of communication cycle calculating between current each calculation server, compares with the half N/2 of total number N of calculation server; If comparative result does not meet pre-conditioned, then enter step 240; If comparative result meets pre-conditioned, then enter step 250;
Step 240, terminates the calculating obtaining collaborative filtering distance submatrix;
Step 230,240 and Fig. 1 to describe S11 and S13 similar, be not described in detail in this.
Step 250, judges whether total number N of calculation server is even number; If total number N of calculation server is even number, then enter step 251; If total number N of calculation server is odd number, then enter step 253;
In embodiments of the present invention, the number of calculation server may in change, may increase and also may reduce, in order to ensure that each computing node does not carry out double counting, there is some difference in the communication mode of even number and odd number, in order to when calculation server number changes, need not reset whole parallel system, then need to carry out step 251 and follow-up judgement.
Step 251, judges whether present communications round p is that last takes turns communication; If current round p is last take turns communication, then enter step 252; If current round p is not last take turns communication, then enter step 253;
Such as calculation server number is 4, then the embodiment of the present invention optimal communication round upper limit is 2 to take turns, after 4 servers carry out steps 220, carry out first round communication process, after calculating after communication, then enter second and take turns communication process, after communication calculates, then step 230 judge round P do not meet design conditions (when P counts from 0, P=0 and 1 according calculation condition, other situations then do not meet design conditions; When P counts from 1, P=1 and 2 according calculation conditions, other situations then do not meet design conditions), enter step 240.
Step 252, each calculation server participating in calculating, judges whether its mark i is less than N/2; If mark i is less than N/2, then enter step 253; If mark i is not less than N/2, then enter step 230;
I=0,1 as previously mentioned ..., so for four calculation servers, be labeled as 0,1,2,3 according to the order of sequence.So take turns communication to last, then 0 and 1 can enter mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server.For 2 and 3, enter the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server, after in parallel computation network, this communication completes, the round of described present communications becomes third time, and so node 2 and 3 just need not calculate collaborative filtering distance submatrix.
Step 253, each calculation server participating in calculating, according to mark i and the round p of current calculation server, carries out modulo operation for N and obtains result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
When described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
Step 260, each calculation server participating in calculating sends local partitioned matrix data to object calculation server, and receives the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Step 270, each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received.
The step 132 of step 260 and step 270 and Fig. 1 and 133 similar, is not described in detail in this.
Reference Fig. 3 and Fig. 4, Fig. 3 are each node communication schematic diagram in the communication process of the embodiment of the present invention two even number calculation server.Fig. 4 is each node communication schematic diagram in the communication process of the embodiment of the present invention two odd number calculation server.
Be the example of 4 calculation servers in Fig. 3, calculation server comprises Node0, Node1, Node2, Node3, and Round1 is first time communication process, and Round2 is first time communication process.
The example of 5 calculation servers in Fig. 4, calculation server comprises Node0, Node1, Node2, Node3, Node4, and Round1 is first time communication process, and Round2 is first time communication process.
Optionally, also comprise: step 270, all collaborative filtering submatrixs that each calculation server participating in calculating calculates are carried out unifying conversion, obtains complete collaborative filtering matrix.
With reference to Fig. 5, it illustrates the structural representation of a kind of system embodiment one for the treatment of collaborative filtering distance of the present invention, specifically can comprise:
N number of calculation server;
Each calculation server participating in calculating comprises:
Digital independent memory module 310, is suitable for each calculation server participating in calculating and reads and store corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Local data computing module 320, is suitable for each calculation server participating in calculating and calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Communication cycle computing module 330, the communication cycle that the calculation server being suitable for each participation calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Described communication cycle computing module comprises:
Object calculation server confirms module 331, is suitable for, for each calculation server participating in calculating, confirming the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Matrix data sending/receiving module 332, is suitable for each calculation server participating in calculating and sends local partitioned matrix data to object calculation server, and receive the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Cooperated computing module 333, is suitable for each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received.
Optionally, described object calculation server confirms that module comprises:
First object calculation server confirms module, be suitable for described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding.
Optionally, described first object calculation server confirms that module comprises:
Stop judge module, being suitable for the round p by carrying out communication cycle calculating between current each calculation server, comparing with the half N/2 of total number N of calculation server;
Second destination server confirms submodule, if the comparative result being suitable for judging meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
First stopping modular, if the comparative result being suitable for judging does not meet pre-conditioned, then terminates the calculating obtaining collaborative filtering distance submatrix.
Optionally, also comprise after described stopping judge module:
Second judge module, is suitable for judging whether total number N of calculation server is even number;
3rd judge module, if the total number N being suitable for calculation server is even number, then judges whether present communications round p is that last takes turns communication;
4th judge module, if being suitable for current round p is that last takes turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2;
3rd destination server confirms submodule, if be suitable for mark i to be less than N/2, then identify the calculation server that i is less than N/2, enter the mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
Proceed to module, if be suitable for mark i to be not less than N/2, then enter the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server;
3rd destination server confirms module, if the total number N being suitable for calculation server is odd number, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server.
Optionally, when described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
Optionally, also comprise:
Sorting module, is suitable for all collaborative filtering submatrixs that each calculation server participating in calculating calculates to carry out unifying conversion, obtains complete collaborative filtering matrix.
Optionally, described matrix data sending/receiving module comprises:
First matrix data sending/receiving module, the communication cycle that the calculation server being suitable for each participation calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
With reference to Fig. 6, it illustrates the schematic flow sheet of a kind of system embodiment two for the treatment of collaborative filtering distance of the present invention, specifically can comprise:
N number of calculation server;
Described each calculation server participating in calculating comprises:
Digital independent memory module 410, is suitable for each calculation server participating in calculating and reads and store corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Local data computing module 420, is suitable for each calculation server participating in calculating and calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Stop judge module 430, being suitable for the round p by carrying out communication cycle calculating between current each calculation server, comparing with the half N/2 of total number N of calculation server; If comparative result does not meet pre-conditioned, then enter the first stopping modular 440; If comparative result meets pre-conditioned, then enter the second judge module 450
First stopping modular 440, terminates the calculating obtaining collaborative filtering distance submatrix;
Second judge module 450, is suitable for judging whether total number N of calculation server is even number; If total number N of calculation server is even number, then enter the 3rd judge module 451; If total number N of calculation server is odd number, then enters the second destination server and confirm submodule 454;
3rd judge module 451, is suitable for judging whether present communications round p is that last takes turns communication; If current round p is last take turns communication, then enter the 4th judge module 452; If current round p is not last take turns communication, then enters the second destination server and confirm submodule 454.
4th judge module 452, if being suitable for current round p is that last takes turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2; If mark i is less than N/2, then enters the second destination server and confirm submodule 453; If mark i is not less than N/2, then enters and stop judge module 430;
Second destination server confirms submodule 453, be suitable for each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server.
Matrix data sending/receiving module 460, is suitable for each calculation server participating in calculating and sends local partitioned matrix data to object calculation server, and receive the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Cooperated computing module 470, is suitable for each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received.
Intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with display at this algorithm provided.Various general-purpose system also can with use based on together with this teaching.According to description above, the structure constructed required by this type systematic is apparent.In addition, the present invention is not also for any certain programmed language.It should be understood that and various programming language can be utilized to realize content of the present invention described here, and the description done language-specific is above to disclose preferred forms of the present invention.
In instructions provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can use with arbitrary array mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that a kind of some or all functions for the treatment of the some or all parts in the equipment of collaborative filtering distance that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the embodiment of the present invention.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
Claims (12)
1., for the treatment of a method for collaborative filtering distance, comprising:
For at least two calculation servers, the calculation server that each participation calculates reads and stores corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Each participation calculates the partitioned matrix data that calculation server calculates this locality, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
The communication cycle that each calculation server participating in calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data; The communication cycle that described each calculation server participating in calculating carries out between each calculation server calculates, and obtains collaborative filtering distance submatrix and specifically comprises:
For each calculation server participating in calculating, confirm the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Each calculation server participating in calculating sends local partitioned matrix data to object calculation server, and receives the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received;
All collaborative filtering submatrixs that each calculation server participating in calculating calculates are carried out unifying conversion, obtains complete collaborative filtering matrix.
2. the method for claim 1, is characterized in that, described for each calculation server participating in calculating, and confirms that the object calculation server that current calculation server is corresponding comprises:
Described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding.
3. method as claimed in claim 2, it is characterized in that, described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm that the object calculation server that current calculation server is corresponding comprises:
By carrying out the round p of communication cycle calculating between current each calculation server, compare with the half N/2 of total number N of calculation server;
If comparative result meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
If comparative result does not meet pre-conditioned, then terminate the calculating obtaining collaborative filtering distance submatrix.
4. method as claimed in claim 3, is characterized in that, describedly will carry out the round p of communication cycle calculating between current each calculation server, also comprises after comparing with the half N/2 of total number N of calculation server:
Judge whether total number N of calculation server is even number;
If total number N of calculation server is even number, then judge whether present communications round p is that last takes turns communication;
If current round p is last take turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2;
If mark i is less than N/2, then identify the calculation server that i is less than N/2, enter the mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
If mark i is not less than N/2, then enters the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server;
If current round p is not last take turns communication, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
If total number N of calculation server is odd number, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server.
5. the method as described in claim 3 or 4, is characterized in that,
When described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
6. the method for claim 1, is characterized in that, described each calculation server participating in calculating communication cycle carried out between each calculation server calculates and comprises:
The communication cycle that each calculation server participating in calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
7., for the treatment of a system for collaborative filtering distance, comprising:
N number of calculation server;
Each calculation server participating in calculating comprises:
Digital independent memory module, is suitable for each calculation server participating in calculating and reads and store corresponding partitioned matrix data; Described partitioned matrix data are from original matrix data, distribute to the partitioned matrix data of described calculation server;
Local data computing module, is suitable for each calculation server participating in calculating and calculates local partitioned matrix data, obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Communication cycle computing module, the communication cycle that the calculation server being suitable for each participation calculating carries out between each calculation server calculates, and obtains the collaborative filtering distance submatrix of local partitioned matrix data;
Described communication cycle computing module comprises:
Object calculation server confirms module, is suitable for, for each calculation server participating in calculating, confirming the object calculation server that current calculation server is corresponding; Described object calculation server is for receiving the matrix data of this locality of current calculation server transmission;
Matrix data sending/receiving module, is suitable for each calculation server participating in calculating and sends local partitioned matrix data to object calculation server, and receive the partitioned matrix data of other calculation servers transmission; Other calculation servers described are the calculation server current calculation server being confirmed as destination server;
Cooperated computing module, is suitable for each calculation server receiving partitioned matrix data, in conjunction with local partitioned matrix data, calculates the collaborative filtering distance submatrix obtained between local partitioned matrix data and the partitioned matrix data received;
Sorting module, is suitable for all collaborative filtering submatrixs that each calculation server participating in calculating calculates to carry out unifying conversion, obtains complete collaborative filtering matrix.
8. system as claimed in claim 7, is characterized in that, described object calculation server confirms that module comprises:
First object calculation server confirms module, be suitable for described for each calculation server participating in calculating, calculate round, the mark of current calculation server and total number of calculation server according to current communication cycle, confirm the object calculation server that current calculation server is corresponding.
9. system as claimed in claim 8, is characterized in that, described first object calculation server confirms that module comprises:
Stop judge module, being suitable for the round p by carrying out communication cycle calculating between current each calculation server, comparing with the half N/2 of total number N of calculation server;
Second destination server confirms submodule, if the comparative result being suitable for judging meets pre-conditioned, then each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the target calculation server of the calculation server corresponding with u as current calculation server;
First stopping modular, if the comparative result being suitable for judging does not meet pre-conditioned, then terminates the calculating obtaining collaborative filtering distance submatrix.
10. system as claimed in claim 9, is characterized in that, also comprise after described stopping judge module:
Second judge module, is suitable for judging whether total number N of calculation server is even number;
3rd judge module, if the total number N being suitable for calculation server is even number, then judges whether present communications round p is that last takes turns communication;
4th judge module, if being suitable for current round p is that last takes turns communication, then each calculation server participating in calculating, judges whether its mark i is less than N/2;
3rd destination server confirms submodule, if be suitable for mark i to be less than N/2, then identify the calculation server that i is less than N/2, enter the mark i according to current calculation server and round p, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server;
Proceed to module, if be suitable for mark i to be not less than N/2, then enter the round p judging to carry out communication cycle calculating between current each calculation server, whether be less than the process of the half N/2 of total number N of calculation server;
3rd destination server confirms module, if the total number N being suitable for calculation server is odd number, then enter each calculation server participating in calculating, according to mark i and the round p of current calculation server, carry out modulo operation for N and obtain result u, using the process of the calculation server corresponding with u as the target calculation server of current calculation server.
11. systems as described in claim 9 or 10, is characterized in that,
When described round counts from 0, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p+1) %N and calculate result u;
When described round counts from 1, the mark i of calculation server identifies from 0, then the described mark i according to current calculation server and round p, carries out modulo operation obtain result u and comprise for N:
By the mark i of current calculation server and round p, substitute into formula (i+p) %N and calculate result u.
12. systems as claimed in claim 7, it is characterized in that, described matrix data sending/receiving module comprises:
First matrix data sending/receiving module, the communication cycle that the calculation server being suitable for each participation calculating adopts message passing interface transmission function MPI_send and message passing interface receiver function MPI_recv to carry out between each calculation server calculates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210518168.2A CN103049489B (en) | 2012-12-05 | 2012-12-05 | For the treatment of the method and system of collaborative filtering distance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210518168.2A CN103049489B (en) | 2012-12-05 | 2012-12-05 | For the treatment of the method and system of collaborative filtering distance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103049489A CN103049489A (en) | 2013-04-17 |
| CN103049489B true CN103049489B (en) | 2015-12-02 |
Family
ID=48062130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210518168.2A Active CN103049489B (en) | 2012-12-05 | 2012-12-05 | For the treatment of the method and system of collaborative filtering distance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103049489B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049486B (en) * | 2012-12-05 | 2015-10-07 | 北京奇虎科技有限公司 | A kind of disposal route of collaborative filtering distance and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999028832A2 (en) * | 1997-12-04 | 1999-06-10 | Microsoft Corporation | Mixtures of bayesian networks |
| CN101082906A (en) * | 2006-05-31 | 2007-12-05 | 中国科学院微电子研究所 | Fixed-base FFT processor with low memory spending and method thereof |
| CN101178720A (en) * | 2007-10-23 | 2008-05-14 | 浙江大学 | Distributed clustering method facing to internet micro-content |
| CN102163226A (en) * | 2011-04-12 | 2011-08-24 | 浙江大学 | Adjacent sorting repetition-reducing method based on Map-Reduce and segmentation |
| CN103049486A (en) * | 2012-12-05 | 2013-04-17 | 北京奇虎科技有限公司 | Processing method and system for synergizing filter distances |
-
2012
- 2012-12-05 CN CN201210518168.2A patent/CN103049489B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999028832A2 (en) * | 1997-12-04 | 1999-06-10 | Microsoft Corporation | Mixtures of bayesian networks |
| CN101082906A (en) * | 2006-05-31 | 2007-12-05 | 中国科学院微电子研究所 | Fixed-base FFT processor with low memory spending and method thereof |
| CN101178720A (en) * | 2007-10-23 | 2008-05-14 | 浙江大学 | Distributed clustering method facing to internet micro-content |
| CN102163226A (en) * | 2011-04-12 | 2011-08-24 | 浙江大学 | Adjacent sorting repetition-reducing method based on Map-Reduce and segmentation |
| CN103049486A (en) * | 2012-12-05 | 2013-04-17 | 北京奇虎科技有限公司 | Processing method and system for synergizing filter distances |
Non-Patent Citations (3)
| Title |
|---|
| 《Distributed Web log Mining Based Collaborative Filtering Recommendation Algorithm》;Xun Wang等;《DCABES 2006 PROCEEDINGS》;20061031;全文 * |
| 《云计算及若干数据挖掘算法的MapReduce化研究》;李军华;《中国优秀硕士学位论文全文数据库》;20110315(第3期);全文 * |
| 《基于大数据集的协同过滤算法的并行化研究》;李改等;《计算机工程与设计》;20120630;第33卷(第6期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103049489A (en) | 2013-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8655949B2 (en) | Correlated information recommendation | |
| US8725673B2 (en) | Evaluating an item based on user reputation information | |
| CN103049486B (en) | A kind of disposal route of collaborative filtering distance and system | |
| CN106547767B (en) | Method and device for determining video cover picture | |
| TWI506577B (en) | System and method for identifying reviewers with incentives | |
| US20140310243A1 (en) | Heart beacon cycle | |
| CN105247507A (en) | Influence score of a brand | |
| CN103886090A (en) | Content recommendation method and device based on user favorites | |
| CN102298650B (en) | Distributed recommendation method of massive digital information | |
| EP3625748B1 (en) | Distributed node cluster for establishing a digital touchpoint across multiple devices on a digital communications network | |
| CN103189836A (en) | Method for classification of objects in a graph data stream | |
| CN110442788A (en) | A kind of information recommendation method and device | |
| Hou et al. | Scaled cluster consensus of discrete-time multi-agent systems with general directed topologies | |
| CN106471497A (en) | Auxiliary using context browses | |
| CN103891245A (en) | Location-aware content detection | |
| CN103019860B (en) | Based on disposal route and the system of collaborative filtering | |
| JP2010533921A5 (en) | ||
| Galletti et al. | A mathematical model of collaborative reputation systems | |
| CN103049489B (en) | For the treatment of the method and system of collaborative filtering distance | |
| CN103049488A (en) | Synergizing filter processing method and system | |
| US9531581B1 (en) | Method and system for identifying and automatically registering domain names | |
| Carchiolo et al. | Users’ attachment in trust networks: reputation vs. effort | |
| Yang et al. | On characterizing and computing the diversity of hyperlinks for anti-spamming page ranking | |
| CN102968459B (en) | Network address processing method and processing device | |
| CN103761024A (en) | Interface display method, device, terminal, server and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220726 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |