CN103049488B - A kind of collaborative filtering disposal route and system - Google Patents

Abstract

The invention discloses a collaborative filtering processing method and system, and relates to the technical field of computers. The method includes: receiving a request for recommending at least one item for a subject, and starting at least two computing servers to perform a recommendation computing process according to the request, and the process includes: each computing server obtains a subject-item weight matrix, a collaborative filtering matrix various block matrix data; each calculation server confirms each calculation server that the current calculation server depends on and the component of each block matrix data; each calculation server sends the dependent component to each calculation server that depends on the component The server; and receiving the components sent by each calculation server; each calculation server recommends at least one item to each local subject to the subject. The present invention can quickly calculate recommended items for huge matrix data, and can reduce the hardware requirements of the computing system, thereby reducing the hardware cost as a whole.

Description

A collaborative filtering method and system

technical field

The invention relates to the field of computer technology, in particular to a collaborative filtering processing method and system.

Background technique

The exponential expansion of information resources on the Internet has brought the so-called "information overload" and "information obsession" problems, that is, it is difficult for people to find the information they are interested in, and even if they find some, it is often mixed with a lot of "noise". Therefore, technologies such as Internet-oriented information retrieval, information filtering and collaborative filtering have emerged. However, information retrieval is not intelligent and cannot learn the interests of users, especially for users with specific professional interests, inputting the same keywords can only get the same retrieval results. Information filtering cannot distinguish the quality of the filtering results of the same topic, and with the rapid increase of information resources, more effective filtering needs to combine people's quality evaluation information. Based on this demand, a recommendation system has emerged. The recommendation system is an intelligent agent system proposed to solve the problem of information overload, which can automatically recommend resources that meet their interests, preferences or needs from a large amount of information to users. With the popularity and rapid development of the Internet, recommender systems have been widely used in various fields, especially in the field of e-commerce, recommender systems have been more and more researched and applied. At present, almost all large e-commerce websites use various forms of recommendation systems to varying degrees, such as Amazon, CDNOW, eBay and Dangdang online bookstores. Among them, collaborative filtering technology has achieved greater success in the current application of recommendation systems.

Collaborative filtering is a class of recommendation algorithms. Collaborative filtering takes into account user evaluation information. Collaborative filtering analyzes user interests, finds similar (interested) users of the specified user in the user group, and synthesizes the evaluation of a certain information by these similar users to form a prediction of the system's preference for the specified user. It is mainly divided into two types: user-based collaborative filtering (Userbased) and product-based (Itembased). For example, based on the user-based collaborative filtering algorithm, its basic idea is: if you want to recommend a product that a user does not have, first calculate the similarity between other users and this user's preferences, and then find this from several users who are most similar to their preferences. Recommend items that the user does not have. Item-based collaborative filtering works similarly.

In the Internet environment, both users and products are massive data, and at the same time due to the effectiveness of the algorithm, there is a certain time requirement. In the existing technology, the calculation of collaborative filtering is basically performed on a computing server. When encountering a large number of For example, when the number of users is on the order of millions, the data of their various matrices is also very large. Using a single server to calculate is too slow and requires relatively high hardware requirements.

Contents of the invention

In view of the above problems, the present invention is proposed to provide a collaborative filtering processing system and a corresponding collaborative filtering processing method that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present invention, a collaborative filtering processing method is provided, including:

A request for recommending at least one item for the subject is received, and at least two computing servers are started according to the request to perform a recommendation computing process, the process including:

For at least two calculation servers, each calculation server participating in the calculation obtains various block matrix data; the block matrix data includes block matrix data of the subject-item weight matrix and block matrix data of the collaborative filtering matrix;

Each calculation server participating in the calculation confirms each server that the current calculation server depends on according to the subject-item weight matrix and the collaborative filtering matrix, and the components of each block matrix data in each dependent calculation server;

Each calculation server participating in the calculation sends the dependent component to each calculation server dependent on the component; and receives the components sent by each calculation server;

For each subject in the block matrix data of the local subject-item weight matrix, each computing server participating in the calculation uses the block matrix data of the local subject-item weight matrix and the block matrix data of the local collaborative filtering matrix and the corresponding relationship between the received components, recommending at least one item to the subject.

According to another aspect of the present invention, a collaborative filtering processing system is provided, including:

a request receiving module and at least two computing servers;

The request receiving module is adapted to receive a request for the subject to recommend at least one item, and start at least two computing servers according to the request;

Each of the at least two computing servers includes:

The block matrix data acquisition module is suitable for each computing server participating in the calculation to obtain various block matrix data; the block matrix data includes the block matrix data of the subject-item weight matrix and the block matrix of the collaborative filtering matrix data;

Dependent calculation server confirmation module, suitable for each calculation server participating in the calculation to confirm each server that the current calculation server depends on and each block in each dependent calculation server according to the subject-item weight matrix and the collaborative filtering matrix the components of the matrix data;

The component sending and receiving module is suitable for each computing server participating in the calculation to send the dependent component to each computing server dependent on the component; and receive the component sent by each computing server;

The recommended item calculation module is suitable for each calculation server participating in the calculation, for each subject in the block matrix data of the local subject-item weight matrix, using the block matrix data of the local subject-item weight matrix and the local collaborative Correspondences between the block matrix data of the filter matrix and the received components recommend at least one item to the subject.

According to a collaborative filtering processing method of the present invention, multiple computing nodes can be used to perform collaborative filtering recommendation calculations in parallel, thereby solving the problems of slow calculation timeliness for huge matrix data and relatively high hardware requirements in the prior art , Highly achieved the beneficial effect of quickly calculating recommended items for the need for fast calculation of recommended items, can quickly calculate recommended items for huge matrix data, and can reduce the hardware requirements of the computing system, which can reduce hardware costs as a whole.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

FIG. 1 shows a schematic flowchart of Embodiment 1 of a collaborative filtering processing method according to an embodiment of the present invention;

Fig. 2 shows a schematic flowchart of Embodiment 2 of a collaborative filtering processing method according to an embodiment of the present invention;

FIG. 3 shows a schematic diagram of a matrix comparison of the Userbased calculation principle according to an embodiment of the present invention;

Fig. 4 shows two computing server matrix comparison examples of the Userbased computing principle according to an embodiment of the present invention;

Fig. 5 shows a schematic flowchart of Embodiment 3 of a collaborative filtering processing method according to an embodiment of the present invention;

FIG. 6 shows a schematic diagram of the calculation principle of Itembased according to an embodiment of the present invention;

Fig. 7 shows a schematic structural diagram of Embodiment 1 of a collaborative filtering processing system according to an embodiment of the present invention;

FIG. 8 shows a schematic structural diagram of Embodiment 2 of a collaborative filtering processing system according to an embodiment of the present invention; and

Fig. 9 shows a schematic structural diagram of Embodiment 3 of a collaborative filtering processing system according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

Referring to FIG. 1 , it shows a schematic flowchart of Embodiment 1 of a collaborative filtering processing method according to the present invention, which may specifically include:

Step 100, receiving a request for recommending at least one item from a subject, and starting at least two computing servers to perform a recommendation computing process according to the request;

In the embodiment of the present invention, the subject may be a user ID in the network, then for various items that the user ID has used or not used in the network, the system or the user may request to recommend a certain item for each user ID Or certain projects, such as recommending related products to users for products purchased on the Internet.

Then, the at least two calculation servers performing the recommendation calculation process include:

Step 110, for at least two calculation servers, each calculation server participating in the calculation obtains various block matrix data; the block matrix data includes the block matrix data of the subject-item weight matrix, and the block matrix data of the collaborative filtering matrix. matrix data;

In the present invention, for the subject-item weight matrix, it may be a user identification-item scoring matrix, such as Table 1:

Table I

In Table 1, if the product category is Item, and the user ID is File1, File2, and File3, then the user corresponding to File1 will score 70, 60, 80, and 90 for Item1 to Item4 used by the user, and Item5 will not be scored; The users of File2 scored 40, 90, and 50 for Item1 to Item3 they had used, Item4 was not scored, and Item5 was 70; users corresponding to File3 scored 70, 80, and 80 for Item2 to Item4 they used, and Item1 and Item5 not scored;

The collaborative filtering matrix is the user similarity matrix for the user in the matrix of Table 1, such as calculating Sim(File1, File2), File1=(70, 60, 80, 90, 0), File2=(40, 90, 50, 0, 70), the sim calculation can be the calculation of the cosine angle value of two vectors, or other functions, that is, a matrix composed of the similarity between the components of each two rows as shown in Table 2, or the item similarity matrix corresponding to the item , that is, the similarity between every two columns.

S _{(File1, File1)} S _{(File1, File2)} S _{(File1, File3)} S _{(File2, File1)} S _{(File2, File2)} S _{(File2, File3)} S _{(File3, File1} S _{(File3, File2)} S _{(File13, File3)}

Table II

Then, when calculating recommended items for users, the matrix in Table 1 and Table 2 can be used for calculation.

In the embodiment of the present invention, for the N computing servers of the parallel computing system, each server can obtain various block matrix data assigned to itself, such as block matrix data including subject-item weight matrix, collaborative filtering matrix The block matrix data for .

In the embodiment of the present invention, the matrix data can send a matrix similar to Table 1 to a calculation server by row blocks, or transpose a matrix similar to Table 1 and send it to a calculation server by column blocks.

Before this step of the embodiment of the present invention, it also includes:

Each calculation server calculates a fact collaborative filtering matrix using the subject-item weight matrix.

In the embodiment of the present invention, there may be N computing servers participating in computing, and N is greater than or equal to 2.

Step 120, each calculation server participating in the calculation confirms each server that the current calculation server depends on according to the subject-item weight matrix and the collaborative filtering matrix, and the components of each block matrix data in each dependent calculation server;

Then each computing server participating in the calculation confirms the components of the block matrix in one or several computing servers that it depends on based on the subject-item weight matrix and the collaborative filtering matrix.

Optionally, each computing server participating in the computing confirms each server that the current computing server depends on according to the subject-item weight matrix and the collaborative filtering matrix, and each block matrix data in the dependent computing servers components, including:

Step S121, each calculation server participating in the calculation obtains the component identification in each block matrix data processed by all other calculation servers;

In the embodiment of the present invention, each calculation server needs to confirm the calculation server it depends on, then it needs to know the row identification of each row component of each matrix (if in the case of the matrix data obtained in step 110 divided by row) , or it is necessary to know the column identification of each column component of each matrix (if the matrix data obtained in step 110 is divided into blocks by column).

Therefore, in the present invention, the server that stores the source matrix data can send the calculation server ID and the corresponding component ID that each process a block matrix to each calculation server that participates in the calculation, or each calculation server itself can send its own The processed component identifier is sent to all other computing servers, and each computing server can call MPI (MessagePassingInterface, message passing interface; a message passing programming interface, and provides a multilingual function library that implements a series of interfaces) to send .

Step S122, each calculation server participating in the calculation is based on the block matrix data in the collaborative filtering distance matrix, and/or the block matrix data of the subject-item weight matrix, and the block matrix data processed by all other calculation servers Middle component identification, confirming the calculation servers that the current calculation server depends on, and the component identifications of the block matrix data in the dependent calculation servers;

If it is based on the userbased situation, each computing server participating in the calculation obtains the block matrix data of the aforementioned user similarity matrix processed by itself, and the block matrix data of the user ID-item subject-item weight matrix, and obtains all other After the components in each block matrix data processed by the computing server are identified, the components in one or several computing servers that it depends on can be confirmed according to the three.

If it is based on Itembased, each computing server participating in the calculation obtains the block matrix data of the aforementioned item similarity matrix processed by itself, and the block matrix data of the user ID-item subject-item weight matrix, and obtains all other computing servers After the components in the processed block matrix data are identified, the components in one or several computing servers that it depends on can be confirmed according to the three.

Step 130, each calculation server participating in the calculation sends the dependent component to each calculation server that depends on the component; and receives the components sent by each calculation server;

After each computing server confirms the components of each block matrix in one or several computing servers that it depends on, the corresponding dependent server sends its dependent component to each computing server that depends on the component.

Optionally, also include:

Step S131, each calculation server participating in the calculation sends the component identification of each block matrix data in each dependent calculation server to each calculation server that the current calculation server depends on;

After each computing server participating in the calculation confirms the computing server it depends on and the components in the computing server, it notifies the computing server it depends on that it needs to send the vectors that the current computing server depends on to the current server. That is, each computing server participating in the calculation sends the component identifiers of the block matrix data in the dependent computing servers to the computing servers that the current computing server depends on.

Wherein, each computing server participating in computing also sends and receives data by invoking MPI when sending and receiving component identifiers.

Further, each calculation server participating in the calculation sends the dependent component to each calculation server that depends on the component; and receiving the components sent by each calculation server includes:

Step S132, each calculation server participating in the calculation sends the corresponding component to each calculation server dependent on the component according to the dependent component identifier; and receives the components sent by each calculation server;

Each computing server participating in the calculation calls MPI to send the dependent component to the computing server that depends on the component, and receives the components sent by other servers.

Step 160, for each subject in the block matrix data of the local subject-item weight matrix, each computing server participating in the calculation utilizes the block matrix data of the local subject-item weight matrix and the block matrix data of the local collaborative filtering matrix. A correspondence between the block matrix data and the received components recommends at least one item to the subject.

After each calculation server participating in the calculation receives the required components, it performs transposition, arrangement, summation and other processing to obtain the final recommendation data.

Referring to FIG. 2 , it shows a schematic flowchart of Embodiment 2 of a collaborative filtering processing method according to the present invention, which may specifically include:

Step 200, receiving a request for the subject to recommend at least one item, and starting at least two computing servers to perform the recommendation computing process according to the request;

In the embodiment of the present invention, the subject may be a user ID in the network, then for various items that the user ID has used or not used in the network, the system or the user may request to recommend a certain item for each user ID Or certain projects, such as recommending related products to users for products purchased on the Internet.

Then, the at least two calculation servers performing the recommendation calculation process include:

Step 210, for at least two servers, each calculation server participating in the calculation obtains various block matrix data; the block matrix data includes user identification-item subject-item weight matrix and user identification-weight mean value matrix block data, block matrix data of user identity similarity matrix;

The embodiment of the present invention is aimed at the specific parallel process of Userbased. In order to facilitate the description of the calculation process of Userbased, the collaborative filtering recommendation calculation process of Userbased is firstly introduced:

Referring to Figure 3, it is a comparison diagram of Userbased calculation matrix. Among them, 201 is the user ID-item subject-item weight matrix R in which the user scores the item, 202 is the transposition matrix R' of R, and 203 is the mean vector of the user scoring the same item, that is, the user ID-weight mean matrix, That is, A, 203 is the transposition matrix A' of A, and 204 is the user identification similarity matrix S, that is, the similarity between every two users. So

The formula for predicting useru and itemi according to the similarity matrix is as follows: where sim(u, u’) is the similarity between users u and u’, which can be calculated according to cosine, pearson coefficient and other algorithms.

$r_{u,i}={\stackrel{\‾}{r}}_u+\frac{{\mathrm{\Σ}}_{u^{,}\∈u}\mathrm{sim}(u,u^{,})(r_{u,i}-{\stackrel{\‾}{r}}_{u^{,}})}{{\mathrm{\Σ}}_{u^{,}\∈u}\mathrm{sim}(u,u^{,})}$ ......Formula 1)

The calculation process is roughly as follows:

1. Obtain the row corresponding to u according to R, find out the item that u has not scored, that is, recommend the complete set I of candidates;

2. Obtain a column corresponding to u' according to R', and obtain the mean value corresponding to u' according to A'. For each item in I, calculate sim(u, u')(r _u'i -avg(r _u' ));

3. According to 2, calculate sim(u, u')(r _u'i -avg(r _u' )) for all u'∈U of u, and sim(u, u'), and sum;

4. According to the mean value corresponding to A to u, and substituting the result of 3, all the predicted values in I can be obtained;

5. According to the requirements, select the items with the highest scores from u to I to get the final recommended item.

For the above description, in step 210, each calculation server participating in the calculation needs to obtain the block data of the user identification-item subject-item weight matrix and the user identification-weight mean matrix assigned to it for processing, and the user identification similarity matrix. Block matrix data.

The block data of the user ID-weight mean matrix can be calculated from the block data of the user ID-item subject-item weight matrix, and can also be pre-calculated. The user ID similarity matrix can also be calculated from the user ID-item subject-item weight matrix.

For convenience of description later, it is preferable to configure the present invention to obtain the block matrix block-by-row in FIG. 3 .

Step 220, each calculation server participating in the calculation obtains the component identification in each block matrix data processed by all other calculation servers;

In the embodiment of the present invention, each calculation server participating in the calculation obtains the row identification of each row component in the user identification-item block matrix in other calculation servers, and the row identification of each user identification-weight block matrix in other calculation servers. Row ID.

Step 230, each calculation server participating in the calculation performs pruning calculation on the block matrix data in the user identification similarity matrix;

Since there may be a large number of users, in order to reduce the impact of users who are not highly related to the users in the current row on the calculation efficiency, in this step, users who are not highly related need to be pruned.

Optionally, this step includes:

Step S11, for each dimension of each row or column of matrix data in the block matrix data in the user identification similarity matrix, sort the values of each dimension, and retain at least one dimension that is ranked first in each row or column .

For example, in the aforementioned S matrix, the first row corresponds to the similarity component of user u ₀ , namely Sim(u ₀ , u ₀ ), Sim(u ₀ , u ₁ )...Sim(u ₀ , u _M ) , then the similarity values of the top n dimensions in the row component can be reserved, and other values can be left blank.

Step 240, each computing server participating in the calculation confirms the current Each computing server that the computing server depends on, and the component identification of each block matrix data in each dependent computing server.

Then for the current computing server, its user similarity block matrix S has all the column identifiers, but only some column values exist in the components of each row, then according to the aforementioned calculation principle, it is necessary to combine the user identifier-project subject-project weight matrix and the user identity-weight mean matrix transposition, even if the user identity-item subject-item weight matrix and the row identity of the user identity-weight mean matrix are transposed, the row vector obtained after the row vector is aligned with the row vector of S, and then it is confirmed that a certain Then, according to the corresponding relationship between the row identifier and the computing server, the computing servers that the current computing server depends on, and the component identifiers of the block matrix data in the dependent computing servers can be confirmed.

Optionally, this step includes:

Step S21, each calculation server participating in the calculation performs row component identification or column component identification transposition of each component identification of the user identification-project subject-item weight matrix and the user identification-weight mean matrix;

Referring to FIG. 4 , it is an example of a computing server and a dependency vector for confirming dependencies according to an embodiment of the present invention. Figure 4 is a schematic diagram of two computing servers, and the data is distributed by rows, as shown above, where the odd rows are assigned to the computing server N0, and the even rows are assigned to the computing server N1. In the relationship matrix S, some data may be discarded due to branch reduction in the calculation, making the matrix a sparse matrix. On the left side of the figure are matrix R and vector A. For the transposition R' and A' used in the calculation, since the data has not changed, only the position of the data has changed, so do not store the data additionally in the calculation, but only perform row-column conversion when fetching the data. For the convenience of description, R and A are placed above the figure in transposed form.

In the embodiment of the present invention, the calculation process of assigning to computing servers by row is described, and the calculation process of assigning to computing servers by columns is similar to the calculation process of assigning to computing servers by rows, which only needs to be transposed, and will not be described in detail here .

As shown in Figure 4, the original user ID-item subject-item weight matrix R includes 5 rows, and each row has a row ID, that is, the row number, which is u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ; user ID- The weight mean matrix A includes and correspondingly includes 5 rows, and each row has a row identifier, that is, the row number, which is u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ; the user similarity matrix S includes 5 columns, and each column has a The column identification, that is, the column number, is u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ;

Then, after N0 obtains the row component identifiers u ₁ and u ₃ of R calculated by N1, all row component identifiers of R are obtained, and after N0 obtains the row component identifiers u ₁ and u ₃ of A calculated by N1, all rows of A are obtained Component identification; then N0 transposes the row component identification (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ )' of R to obtain (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ); The row components of A are marked (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ )', and transposed to obtain (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ).

Step S22, aligning the result obtained by transposing the row component identification with the column component identification of the user identification similarity matrix, or aligning the result obtained by transposing the column component identification with the row component identification of the co-user identification similarity matrix ;

The result obtained by transposing the row component identifiers of R and A is aligned with the column component identifiers (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ) of S.

Step S23, for the currently reserved dimensions of each row or column, mark the row component identifier or column component identifier of the corresponding reserved dimension;

Step S24, according to the row component identification or column component identification of the mark, and the row component identification or column component identification in the block matrix data of the user identification-item subject-item weight matrix and user identification-weight mean matrix obtained locally Compare and judge the row component ID or column component ID that does not exist locally;

Step S25, according to the computing server to which the row component ID or column component ID that does not exist locally, confirm the computing servers that the current server depends on, and the user ID-item subject-item weight matrix and user ID-weight in each dependent computing server Component identification of the mean matrix.

For computing server N0, the row 1, row 3, and row 5 in the S matrix it processes are reserved for each row (u ₀ , u ₁ , u ₂ , -, u ₄ ), then the R that needs to be calculated The row vectors in the matrix and the A matrix are u ₀ , u ₁ , u ₂ , and u ₄ rows, and it can be known from the above that u ₁ is in N1, so N0 depends on the row component u ₁ in N1.

For the computing server N0, the row 1, row 3, and row 5 in the S matrix it processes are reserved for each row (-, u ₁ , u ₂ , u ₃ , u ₄ ), then the R that needs to be calculated The row vectors in the matrix and the A matrix are u ₁ , u ₂ , u ₃ , and u ₄ rows, and it can be known from the above that u ₂ and u ₄ are in N0, then N1 depends on the row components u ₂ , u 4 in N0 u ₄ .

Step 250, each calculation server participating in the calculation sends the component identification of each block matrix data in each dependent calculation server to each calculation server that the current calculation server depends on;

Then N0 notifies N1 that u ₁ line components need to be sent to N0, and N1 notifies N0 that u ₂ and u ₄ line components need to be sent to N1.

Step 260, each calculation server participating in the calculation sends the corresponding component to each calculation server dependent on the component according to the dependent component identifier; and receives the components sent by each calculation server;

As mentioned above, N0 sends (u ₂ , u ₄ ) to N1, and N1 receives (u ₂ , u ₄ ); N1 sends (u ₁ ) to N0, and N0 receives (u ₁ ).

Step 270, for each subject in the block matrix data of the local subject-item weight matrix, each computing server participating in the calculation uses the block matrix data of the local subject-item weight matrix and the block matrix data of the local collaborative filtering matrix. A correspondence between the block matrix data and the received components recommends at least one item to the subject.

For each subject in the block matrix data of the local subject-item weight matrix, each computing server participating in the calculation uses the block matrix data of the local subject-item weight matrix and the block matrix data of the local collaborative filtering matrix and the correspondence between the received components, recommending at least one item to the subject

Then each computing node can calculate the recommended item for each user ui in the current computing node according to the formula (1). That is, N0 calculates recommended items for u ₀ , u ₁ , u ₃ , and N1 calculates recommended items for u ₂ , u ₄ .

Wherein, the computing servers send and receive data by calling the message transfer interface.

This embodiment is a preferred embodiment for the Userbased situation, and the sequence of some steps may be changed according to specific situations, and no limitation is imposed here.

Referring to FIG. 5 , it shows a schematic flowchart of Embodiment 3 of a collaborative filtering processing method according to the present invention, which may specifically include:

Step 300, receiving a request for the subject to recommend at least one item, and starting at least two computing servers to perform the recommendation computing process according to the request;

In the embodiment of the present invention, the subject may be a user ID in the network, then for various items that the user ID has used or not used in the network, the system or the user may request to recommend a certain item for each user ID Or certain projects, such as recommending related products to users for products purchased on the Internet.

Then, the at least two calculation servers performing the recommendation calculation process include:

Step 310, for the N calculation servers, each calculation server participating in the calculation obtains various block matrix data; the block matrix data includes the block matrix data of the user identification-item subject-item weight matrix, item-item block matrix data for similarity matrix;

Referring to FIG. 6 , it shows a schematic diagram of the calculation principle of Itembased according to the embodiment of the present invention. Figures 301 and 302 are the matrix R of user identification-item subject-item weight, and figure 303 is the matrix S of item-item similarity.

Its calculation formula is as follows:

$r_{u,i}={\stackrel{\‾}{r}}_u+\frac{{\mathrm{\Σ}}_{i^{,}\∈I}\mathrm{sim}(i,i^{,})r_{u,i^{,}}}{{\mathrm{\Σ}}_{i^{,}\∈I}\mathrm{sim}(i,i^{,})}$ ...Formula (2)

The calculation process is as follows:

1. Obtain the row corresponding to u according to R, find out the item that u has not scored, that is, recommend the complete set I of candidates;

2. For each i in I, get a column of the sim matrix, get a set of items similar to i, and sum them up according to the formula to get the predicted scoring value;

3. According to the requirements, select the items with the highest scores from u to I to get the final recommended item.

In the embodiment of the present invention, the item-item similarity matrix S can be calculated from the user identification-item subject-item weight matrix R.

For the convenience of description, in the embodiment of the present invention, the matrix is divided into rows and provided to each computing server for calculation. The process of dividing the matrix into columns and providing it to each calculation server for calculation is the same as the above-mentioned row-based division into blocks. The calculation principle and process of are similar, only need to carry out the corresponding transposition calculation, which will not be described in detail in this and subsequent steps.

Step 320, each calculation server participating in the calculation obtains the component identification in each block matrix data processed by all other calculation servers;

That is, each computing server sends the obtained identifiers of each row in the S block matrix to other N-1 servers.

Step 330, each calculation server participating in the calculation calculates the candidate recommendation set according to the user identification-item subject-item weight matrix;

Each calculation server participating in the calculation obtains a row corresponding to u according to the current R block matrix, and finds out the item that u has not scored, that is, the candidate recommendation set I of the recommendation candidate.

Step 340, each calculation server participating in the calculation confirms the calculation servers that the current calculation server depends on according to the candidate recommendation set, the block matrix data of the item-item similarity matrix and the user identification-item subject-item weight matrix, and the The component identification of each block matrix data in each dependent calculation server;

Then the vector corresponding to the row identifier of the entire S matrix can be transposed, aligned with the column identifier of the R matrix, and according to the dimensions of the corresponding candidate recommendation set I in the R matrix, it is confirmed which row components of the S matrix are needed, and where the row components are which computing server.

Optionally, this step includes:

Step S31, each calculation server participating in the calculation performs row component identification or column component identification transposition of each component identification of the item-item similarity matrix;

For example, computing server N0 obtains u ₀ , u ₂ , u ₄ rows of R, and i ₀ , i ₂ , i ₄ rows of S, and computing server N1 obtains u ₁ , u ₃ rows of R and i ₁ , i ₃ of S. row, then after N0 obtains the row component identifier of N1, it transposes the vector (i ₀ , i ₁ , i ₂ , i ₃ , i ₄ )' corresponding to each row component identifier.

Step S32, aligning the result obtained by transposing the row component identification with the column component identification of the user identification-item subject-item weight matrix, or aligning the result obtained by transposing the column component identification with the co-user identification similarity matrix The line components of the mark are aligned;

The calculation server N0 aligns the transposed result of the row component identification of S with a row of the R matrix, that is, with each column component (u ₀ , u ₁ , u ₂ , u ₃ , u ₄ ) of R.

Step S33, according to the row component identifier or column component identifier of the mark, and the row component identifier or column component identifier in the block matrix data of the user identifier-item subject-item weight matrix and user identifier-weight mean matrix obtained locally Compare and judge the row component ID or column component ID that does not exist locally;

For example, for row u ₀ in N0, its corresponding recommendation candidate set is (i ₂ , i ₃ , i ₄ ), then i ₃ is in N1; for row u ₂ , its corresponding recommendation candidate set is (i ₂ , i ₃ , i ₄ ), then i ₃ is in N1, and for row u ₄ , the corresponding recommendation candidate set is (i ₀ , i ₃ , i ₄ ), then i ₃ is in N1.

For row u ₁ in N1, its corresponding recommendation candidate set is (i ₁ , i ₃ , i ₄ ), then i ₄ is in N0; for u ₃ row, its corresponding recommendation candidate set is (i ₀ , i ₃ , i ₄ ), then i ₀ and i ₄ are in N0;

Step S34, according to the computing server to which the row component ID or column component ID that does not exist locally, confirm the computing servers that the current server depends on, and the user ID-item subject-item weight matrix and user ID-weight in each dependent computing server Component identification of the mean matrix.

Then N0 depends on i ₃ of N1, and N1 depends on i ₀ and i ₄ of N0.

Step 350, each calculation server participating in the calculation sends the component identification of each block matrix data in each dependent calculation server to each calculation server that the current calculation server depends on;

Then N0 notifies N1 to send i ₃ row vectors to N0, and N1 notifies N0 to send i ₀ , i ₄ row vectors to N1.

Step 360, each calculation server participating in the calculation sends the corresponding component to each calculation server dependent on the component according to the dependent component identifier; and receives the components sent by each calculation server;

Then N1 sends i ₃ row vectors to N0, and N0 sends i ₀ , i ₄ row vectors to N1.

Step 370, for each subject in the block matrix data of the local subject-item weight matrix, each computing server participating in the calculation utilizes the block matrix data of the local subject-item weight matrix and the block matrix data of the local collaborative filtering matrix. A correspondence between the block matrix data and the received components recommends at least one item to the subject.

Then according to the formula (2), the recommended items for each u _i are calculated according to the local weight block matrix data, collaborative filtering block matrix data and received components.

This embodiment is a preferred embodiment for the case of Itembased, and the order of certain steps may be changed according to the actual situation, which is not limited here.

Referring to FIG. 7, it shows a schematic structural diagram of Embodiment 1 of a collaborative filtering processing system according to the present invention, which may specifically include:

request receiving module 700 and at least two computing servers;

The request receiving module 700 is adapted to receive a request for recommending at least one item for the subject, and start at least two computing servers according to the request;

Each of the at least two computing servers includes:

The block matrix data acquisition module 710 is suitable for at least two computing servers, and each computing server participating in the calculation obtains various block matrix data; the block matrix data includes the block matrix data of the subject-item weight matrix , the block matrix data of the collaborative filtering matrix;

Dependent calculation server confirmation module 720, adapted to each calculation server participating in the calculation, according to the subject-item weight matrix and the collaborative filtering matrix to confirm each server that the current calculation server depends on, and each component in each dependent calculation server the components of the block matrix data;

The component sending and receiving module 730 is adapted for each computing server participating in the computing to send the dependent component to each computing server dependent on the component; and receive the components sent by each computing server;

The recommended item calculation module 740 is suitable for each calculation server participating in the calculation, for each subject in the block matrix data of the local subject-item weight matrix, using the block matrix data of the local subject-item weight matrix, the local A correspondence between the block matrix data of the collaborative filtering matrix and the received components recommends at least one item to the subject.

Optionally, the dependent computing server confirmation module includes:

The component identification acquisition module is suitable for each calculation server participating in the calculation to obtain the component identification in each block matrix data processed by all other calculation servers;

The first dependent server confirmation module is suitable for each calculation server participating in the calculation according to the block matrix data in the collaborative filtering distance matrix, and/or the block matrix data of the subject-item weight matrix, and all other calculation server processing The component identifiers in each block matrix data of the current calculation server are confirmed, and the component identifiers of each block matrix data in the dependent calculation servers are confirmed.

Optionally, also include:

The identification sending module is suitable for each calculation server participating in the calculation to send the component identification of each block matrix data in each calculation server on which it depends, to each calculation server on which the current calculation server depends;

Further, the component sending and receiving module includes:

The first component sending and receiving module is adapted for each computing server participating in computing to send the corresponding component to each computing server dependent on the component according to the dependent component identifier; and receive the components sent by each computing server.

Optionally, the subject-item weight matrix includes: user identification-item subject-item weight matrix and user identification-weight mean matrix;

The collaborative filtering matrix is a user identification similarity matrix corresponding to the user identification-item subject-item weight matrix;

Further, the components include the components of the user identification-item subject-item weight matrix and the user identification-weight mean matrix components.

Optionally, the first dependent server confirmation module includes:

A pruning module, adapted to perform pruning calculations on the block matrix data in the user identification similarity matrix for each calculation server participating in the calculation;

The second dependent server confirmation module is suitable for each computing server participating in the calculation according to the block matrix data in the user identification similarity matrix after the pruning, and the block matrix data processed by all other computing servers The middle component identification is to confirm the calculation servers that the current calculation server depends on, and the component identifications of the block matrix data in the dependent calculation servers.

Optionally, the pruning module includes:

The first pruning module is suitable for sorting the values of each dimension for each dimension of each row or column of matrix data in the block matrix data in the user identification similarity matrix, and retaining the sorting order in each row or column At least one dimension before .

Optionally, the second dependent server confirmation module includes:

The first row/column component identification transposition module is suitable for each calculation server participating in the calculation to perform row component identification or column component identification conversion of each component identification of the user identification-project subject-item weight matrix and the user identification-weight mean matrix set;

The first row/column component identification alignment module is suitable for aligning the result obtained by the row component identification transposition with the column component identification of the user identification similarity matrix, or aligning the result obtained by the column component identification transposition with the coordinating The row component identification alignment of the user identification similarity matrix;

The first reservation module is adapted to mark the row component identifier or the column component identifier of the corresponding reserved dimension for the dimension reserved for each row or column at present;

The first judging module is adapted to use the tagged row component ID or column component ID with the row component ID in the block matrix data of the locally acquired user ID-item subject-item weight matrix and user ID-weight mean matrix or column component identifier comparison, to determine the row component identifier or column component identifier that does not exist locally;

The third dependent server confirmation module is adapted to confirm the computing servers that the current server depends on, and the user ID-project subject-project weight in the dependent computing servers according to the computing server to which the row component identifier or column component identifier that does not exist locally matrix and userid - Component identity of the weight mean matrix.

Optionally, the subject-item weight matrix includes: user identification-item subject-item weight matrix;

The collaborative filtering matrix is an item-item similarity matrix corresponding to the user identification-item subject-item weight matrix;

Further, the components include components of an item-item similarity matrix.

Optionally, the dependent server confirmation module includes:

The first candidate recommendation set calculation module is suitable for each calculation server participating in the calculation to calculate the candidate recommendation set according to the user identification-item subject-item weight matrix;

The fourth dependent server confirmation module is suitable for each calculation server participating in the calculation to confirm each calculation server that the current calculation server depends on according to the candidate recommendation set, the block matrix data of the item-item similarity matrix and the user identification-item subject-item weight matrix. The computing server, and the component identifiers of the block matrix data in the dependent computing servers.

Optionally, the fourth dependent server confirmation module includes:

The second row/column component identification transposition module is suitable for each calculation server participating in the calculation to perform row component identification or column component identification transposition for each component identification of the item-item similarity matrix;

The second row/column component identification alignment module is adapted to align the result obtained after the row component identification is transposed with the column component identification of the user identification-item subject-item weight matrix, or to align the result obtained after the column component identification is transposed The result is aligned with the row component identification of the co-user identification similarity matrix;

The second judging module is adapted to use the row component identifier or column component identifier of the mark, and the row component identifier in the block matrix data of the locally acquired user identifier-item subject-item weight matrix and user identifier-weight mean matrix or column component identifier comparison, to determine the row component identifier or column component identifier that does not exist locally;

The fifth dependent server confirmation module is adapted to confirm the calculation servers that the current server depends on and the user identification-project subject-item weight in each dependent calculation server according to the calculation server to which the row component identifier or column component identifier that does not exist locally belongs matrix and userid - Component identity of the weight mean matrix.

Optionally, the computing servers send and receive data by calling a message transfer interface.

Referring to FIG. 8 , it shows a schematic structural diagram of Embodiment 2 of a collaborative filtering processing system according to an embodiment of the present invention, which may specifically include:

request receiving module 800 and at least two computing servers;

The request receiving module 800 is adapted to receive a request for recommending at least one item for the subject, and start at least two computing servers according to the request;

Each of the at least two computing servers includes:

The first block matrix data acquisition module 810 is suitable for each computing server participating in the calculation to obtain various block matrix data; the block matrix data includes user identification-item subject-item weight matrix and user identification-weight mean value block data of matrix, block matrix data of user identity similarity matrix;

The first component identification acquisition module 820 is adapted for each calculation server participating in the calculation to obtain the component identification in each block matrix data processed by all other calculation servers;

The pruning module 830 is adapted to perform pruning calculations on the block matrix data in the user identification similarity matrix for each calculation server participating in the calculation;

The second dependent server confirmation module 840 is suitable for each computing server participating in the calculation according to the block matrix data in the user identification similarity matrix after the pruning, and the block matrix data processed by all other computing servers The component identification in the data, confirming the calculation servers that the current calculation server depends on, and the component identification of each block matrix data in the dependent calculation servers;

The first identification sending module 850 is suitable for each calculation server participating in the calculation to send the component identification of each block matrix data in each calculation server on which it depends to each calculation server that the current calculation server depends on;

The first component sending and receiving module 860 is adapted for each computing server participating in the calculation to send the corresponding component to each computing server dependent on the component according to the dependent component identifier; and receive the components sent by each computing server;

The first recommended item calculation module 870 is adapted to use the local block matrix data of the subject-item weight matrix for each subject in the block matrix data of the local subject-item weight matrix, A correspondence between block matrix data of the local collaborative filtering matrix and the received components recommends at least one item to the subject.

Referring to FIG. 9 , it shows a schematic structural diagram of Embodiment 3 of a collaborative filtering processing system according to an embodiment of the present invention, which may specifically include:

request receiving module 900 and at least two computing servers;

The request receiving module 900 is adapted to receive a request for recommending at least one item for the subject, and start at least two computing servers according to the request;

Each of the at least two computing servers includes:

The second block matrix data acquisition module 910 is suitable for each computing server participating in the calculation to obtain various block matrix data; each block matrix data includes user identification-item subject-item weight matrix and user identification-weight mean value block data of matrix, block matrix data of user identity similarity matrix;

The second component identification acquisition module 920 is adapted for each calculation server participating in the calculation to obtain the component identification in each block matrix data processed by all other calculation servers;

The first candidate recommendation set calculation module 930 is suitable for each calculation server participating in the calculation to calculate the candidate recommendation set according to the user identification-item subject-item weight matrix;

The fourth dependent server confirmation module 940 is suitable for each calculation server participating in the calculation to confirm the current calculation server depends on according to the candidate recommendation set, the block matrix data of the item-item similarity matrix and the user identification-item subject-item weight matrix Each calculation server, and the component identification of each block matrix data in each calculation server that depends on;

The second identification sending module 950 is suitable for each calculation server participating in the calculation to send the component identification of each block matrix data in each calculation server it depends on to each calculation server that the current calculation server depends on;

The second component sending and receiving module 960 is adapted for each computing server participating in the calculation to send the corresponding component to each computing server dependent on the component according to the dependent component identifier; and receive the components sent by each computing server;

The second recommended item calculation module 970 is suitable for each calculation server participating in the calculation, for each subject in the block matrix data of the local subject-item weight matrix, using the block matrix data of the local subject-item weight matrix, A correspondence between block matrix data of the local collaborative filtering matrix and the received components recommends at least one item to the subject.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or in the description of the alignment. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the collaborative filtering processing device according to the embodiment of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

This paper discloses A1, a collaborative filtering processing method, including: receiving a request for recommending at least one item for a subject, and starting at least two computing servers to perform a recommendation computing process according to the request, and the process includes: targeting at least two computing servers , each calculation server participating in the calculation obtains various block matrix data; the block matrix data includes the block matrix data of the subject-item weight matrix and the block matrix data of the collaborative filtering matrix; each calculation participating in the calculation According to the subject-item weight matrix and the collaborative filtering matrix, the server confirms each server that the current calculation server depends on, and the components of each block matrix data in each dependent calculation server; each calculation server participating in the calculation will The above-mentioned dependent component is sent to each calculation server that depends on the component; and the component sent by each calculation server is received; each calculation server participating in the calculation is directed at each subject in the block matrix data of the local subject-item weight matrix, using The local block matrix data of the subject-item weight matrix, the block matrix data of the local collaborative filtering matrix and the corresponding relationship between the received components recommend at least one item to the subject. A2. The method as described in A1, each of the computing servers participating in the computing confirms the servers that the current computing server depends on according to the subject-item weight matrix and the collaborative filtering matrix, and the computing servers that the computing servers depend on The components of each block matrix data include: each computing server participating in the calculation obtains the component identifiers in each block matrix data processed by all other computing servers; each computing server participating in the calculation according to the local collaborative filtering distance matrix Block matrix data, block matrix data of the subject-item weight matrix, and component identifiers in each block matrix data processed by all other computing servers, confirm the computing servers that the current computing server depends on, and the dependent computing servers Component IDs for each block matrix data for . A3. The method as described in A2, further comprising: each computing server participating in the calculation sends the component identification of each block matrix data in each computing server it depends on to each computing server that the current computing server depends on; further , each computing server participating in the calculation sends the dependent component to each computing server that depends on the component; and receiving the components sent by each computing server includes: each computing server according to the dependent component identifier, sending The corresponding component is sent to each computing server depending on the component; and the component sent by each computing server is received. A4, the method as described in A3, the subject-item weight matrix includes: user identification-item subject-item weight matrix and user identification-weight mean matrix; the collaborative filtering matrix is the corresponding user identification-project subject-item weight The user ID similarity matrix of the matrix; further, the components include the components of the user ID-item subject-item weight matrix and the user ID-weight mean matrix components. A5, the method as described in A4, each of the computing servers participating in the calculation is based on the block matrix data in the local collaborative filtering distance matrix, the block matrix data of the subject-item weight matrix, and all other computing servers. The component identification in each block matrix data, confirming the calculation servers that the current calculation server depends on, and the component identification of each block matrix data in the dependent calculation servers include: each calculation server participating in the calculation for the user Identify the block matrix data in the similarity matrix to perform pruning calculations; each computing server participating in the calculation identifies the block matrix data in the similarity matrix according to the pruned user, and all other computing servers process The component identifiers in each block matrix data identify the computing servers that the current computing server depends on, and the component identifiers of each block matrix data in the dependent computing servers. A6. The method as described in A5, wherein each calculation server participating in the calculation performs pruning calculation for the block matrix data in the user identification similarity matrix including: for each of the block matrix data in the user identification similarity matrix For each dimension of the row or column matrix data, the values of each dimension are sorted, and at least one dimension that is ranked first in each row or column is reserved. A7. The method as described in A6, each of the computing servers participating in the computing is based on the block matrix data in the user identification similarity matrix after the pruning, and the blocks processed by all other computing servers The component identification in the matrix data, confirming the calculation servers that the current calculation server depends on, and the component identification of each block matrix data in the dependent calculation servers include: each calculation server participating in the calculation will user identification - project subject - Each component identification of the item weight matrix and the user identification-weight mean matrix is subjected to row component identification or column component identification transposition; the result obtained after the transposition of the row component identification is aligned with the column component identification of the user identification similarity matrix, or will be obtained by The result obtained after the column component identification is transposed is aligned with the row component identification of the co-user identification similarity matrix; for the dimensions reserved for each current row or column, mark the row component identification or column component identification of the corresponding reserved dimension; according to the label Compared with the row or column component identifiers in the block matrix data of the user identifier-item subject-item weight matrix and user identifier-weight mean matrix obtained locally, it is judged that there is no local Row component ID or column component ID; according to the calculation server to which the row component ID or column component ID does not exist locally, confirm the calculation servers that the current server depends on, and the user ID-project subject-item weight matrix in each dependent calculation server and the component ids of the user id-weight mean matrix. A8. The method as described in A3, the subject-item weight matrix includes: user identification-item subject-item weight matrix; the collaborative filtering matrix is an item-item similarity matrix corresponding to the user identification-item subject-item weight matrix ; Further, the components include components of an item-item similarity matrix. A9, the method as described in A8, each of the computing servers participating in the calculation is based on the block matrix data in the local collaborative filtering distance matrix, the block matrix data of the subject-item weight matrix, and all other computing servers. The component identification in each block matrix data, confirming the calculation servers that the current calculation server depends on, and the component identification of each block matrix data in the dependent calculation servers include: each calculation server participating in the calculation according to the user identification - The project subject-project weight matrix calculates the candidate recommendation set; each computing server participating in the calculation confirms that the current computing server relies on Each computing server of , and the component identification of each block matrix data in each dependent computing server. A10, the method as described in A9, each of the computing servers participating in the computing confirms that the current computing server depends on the Each calculation server of each calculation server, and the component identification of each block matrix data in each dependent calculation server includes: each calculation server participating in the calculation performs row component identification or column identification of each component identification of the item-item similarity matrix Component identification transposition; align the result obtained by the row component identification with the column component identification of the user identification-item subject-item weight matrix, or align the result obtained by the column component identification with the co-user identification The row component identification of the similarity matrix is aligned; according to the row component identification or column component identification of the mark, the row component in the block matrix data of the user identification-item subject-item weight matrix and the user identification-weight mean matrix obtained locally ID or column component ID comparison to determine the row component ID or column component ID that does not exist locally; according to the computing server to which the row component ID or column component ID that does not exist locally belongs, confirm the computing servers that the current server depends on, and the dependent computing servers. Compute the component IDs of the user ID-item subject-item weight matrix and the user ID-weight mean matrix in the server. A11. The method described in any one of A1-A10, wherein the computing servers send and receive data by calling a message transfer interface.

This paper discloses B12, a collaborative filtering processing system, including: a request receiving module and at least two computing servers; the request receiving module is adapted to receive a request for recommending at least one item for the subject, and start at least two according to the request Computing server; each of the computing servers in at least two computing servers includes: a block matrix data acquisition module, which is suitable for each computing server participating in the calculation to obtain various block matrix data; each of the block matrix data includes a main body - block matrix data of the item weight matrix, block matrix data of the collaborative filtering matrix; rely on the calculation server confirmation module, suitable for each calculation server participating in the calculation to confirm the current calculation according to the subject-item weight matrix and the collaborative filtering matrix Each server that the server depends on, and the components of each block matrix data in the dependent computing servers; the component sending and receiving module is suitable for each computing server participating in the calculation to send the dependent component to the dependent component each calculation server; and receive the components sent by each calculation server; the recommended item calculation module is suitable for each calculation server participating in the calculation for each subject in the block matrix data of the local subject-item weight matrix, using the local subject - a correspondence between the block matrix data of the item weight matrix, the block matrix data of the local collaborative filtering matrix and the received components, recommending at least one item to said subject. B13, the system as described in B12, said dependent calculation server confirmation module includes: component identification acquisition module, suitable for each calculation server participating in the calculation to obtain the component identification in each block matrix data processed by other all calculation servers; the first Relying on the server confirmation module, suitable for each computing server participating in the calculation according to the block matrix data in the collaborative filtering distance matrix, and/or the block matrix data of the subject-item weight matrix, and all other computing servers. The component identifiers in the block matrix data confirm the computing servers that the current computing server depends on, and the component identifiers of the block matrix data in the dependent computing servers. B14. The system as described in B13, further comprising: an identification sending module, which is suitable for each calculation server participating in the calculation to send the component identification of each block matrix data in each calculation server on which the current calculation server depends. Each computing server; further, the component sending and receiving module includes: a first component sending and receiving module, adapted for each computing server participating in the calculation to send the corresponding component to the dependent component according to the dependent component identifier component to each computing server; and receive the components sent by each computing server. B15, the system as described in B14, the subject-item weight matrix includes: user identification-item subject-item weight matrix and user identification-weight mean matrix; the collaborative filtering matrix is the corresponding user identification-item subject-item weight The user ID similarity matrix of the matrix; further, the components include the components of the user ID-item subject-item weight matrix and the user ID-weight mean matrix components. B16, the system as described in B15, the first dependent server confirmation module includes: a pruning module, adapted to perform pruning calculations for the block matrix data in the user identification similarity matrix for each calculation server participating in the calculation; The second dependent server confirmation module is suitable for each computing server participating in the calculation according to the block matrix data in the user identification similarity matrix after the pruning, and the block matrix data processed by all other computing servers The middle component identification is to confirm the calculation servers that the current calculation server depends on, and the component identifications of the block matrix data in the dependent calculation servers. B17. The device as described in B16, the pruning module includes: a first pruning module, suitable for each dimension of each row or column of matrix data in the block matrix data in the user identification similarity matrix, each Dimension values are sorted, and at least one dimension that ranks first in each row or column is retained. B18, the system as described in B17, the second dependent server confirmation module includes: the first row/column component identification transposition module, suitable for each calculation server participating in the calculation user identification-item subject-item weight matrix and Each component identification of the user identification-weight mean matrix is subjected to row component identification or column component identification transposition; the first row/column component identification alignment module is suitable for transposing the result obtained by the row component identification and the user identification similarity matrix Align the column component identification of the column component identification, or align the result obtained after the transposition of the column component identification with the row component identification of the co-user identification similarity matrix; the first reservation module is suitable for the dimensions reserved for each current row or column, and the corresponding marking The row component identifier or column component identifier of the reserved dimension; the first judgment module is adapted to combine the locally acquired user identifier-item subject-item weight matrix and user identifier-weight mean value according to the marked row component identifier or column component identifier Comparing the row component identifiers or column component identifiers in the block matrix data of the matrix, judging the row component identifiers or column component identifiers that do not exist locally; the third depends on the server confirmation module, which is suitable for judging the row component identifiers or column component identifiers that do not exist locally Identify the computing server to which the current server depends, and confirm the computing servers that the current server depends on, and the component identifications of the user ID-project subject-item weight matrix and the user ID-weight mean matrix in each dependent computing server. B19, the system as described in B14, is characterized in that, described main body-item weight matrix comprises: user identification-item main body-item weight matrix; Described collaborative filtering matrix is the item of corresponding user identification-item main body-item weight matrix - Item similarity matrix; further, the components include items of item-item similarity matrix. B20, the system as described in B19, said dependent server confirmation module includes: a first candidate recommendation set calculation module, suitable for each calculation server participating in the calculation to calculate the candidate recommendation set according to the user identification-item subject-item weight matrix; Four dependent server confirmation modules, suitable for each calculation server participating in the calculation to confirm each calculation that the current calculation server depends on according to the candidate recommendation set, the block matrix data of the item-item similarity matrix and the user identification-item subject-item weight matrix The server, and the component identification of each block matrix data in each dependent calculation server. B21, the system as described in B20, the fourth dependent server confirmation module includes: the second row/column component identification transposition module, which is suitable for each component of the item-item similarity matrix for each calculation server participating in the calculation The logo performs row component logo or column component logo transposition; the second row/column component logo alignment module is suitable for combining the result obtained after the row component logo transposition with the column component logo of the user logo-item subject-item weight matrix Align, or align the result obtained by the column component identifier transposition with the row component identifier of the co-user identifier similarity matrix; the second judging module is suitable for matching the locally acquired Compare the row component ID or column component ID in the block matrix data of the user ID-item subject-item weight matrix of the user ID-weight mean matrix, and judge the row component ID or column component ID that does not exist locally; the fifth depends on the server The confirmation module is adapted to confirm the calculation servers that the current server depends on, and the user identification-project subject-item weight matrix and user identification in each dependent calculation server according to the calculation server to which the row component identifier or column component identifier that does not exist locally - Component identification of the weight mean matrix. B22. The system according to one of B12-B21, wherein the computing servers send and receive data by calling a message transfer interface.

Claims (20)

1. a collaborative filtering disposal route, comprising:

Receive the request recommending at least one project for main body, start at least two calculation servers according to described request and carry out recommendation computation process, described process comprises:

For at least two calculation servers, the calculation server that each participation calculates obtains various partitioned matrix data; Described each partitioned matrix data comprise main body-Term Weight partitioning of matrix matrix data, collaborative filtering partitioning of matrix matrix data;

Each calculation server participating in calculating is according to the main body-Term Weight matrix of this locality and collaborative filtering matrix, component identity with the main body-Term Weight matrix of other calculation servers, confirm each server that current calculation server relies on, and the component of each partitioned matrix data in each calculation server of described dependence; Wherein, described component identity comprises row component identity and row component identity;

Each calculation server participating in calculating is sent to each calculation server of this component of dependence by described by the component relied on; And receive the component of each calculation server transmission;

Each calculation server participating in calculating is for each main body in the main body-Term Weight partitioning of matrix matrix data of this locality, main body-Term Weight partitioning of matrix the matrix data of utilization this locality, the corresponding relation between local collaborative filtering partitioning of matrix matrix data and the component received, give described main body by least one project recommendation.

2. the method for claim 1, described each calculation server participating in calculating confirms according to described main body-Term Weight matrix and collaborative filtering matrix each server that current calculation server relies on, and the component of each partitioned matrix data in each calculation server of described dependence, comprising:

Each calculation server participating in calculating obtains component identity in each partitioned matrix data of other all calculation server process;

Each calculation server participating in calculating is according to the partitioned matrix data in the collaborative filtering distance matrix of this locality, main body-Term Weight partitioning of matrix matrix data, with component identity in each partitioned matrix data of other all calculation server process, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

3. method as claimed in claim 2, also comprises:

Each calculation server participating in calculating, by the component identity of each partitioned matrix data in relied on each calculation server, sends to each calculation server that current calculation server relies on;

Further, each calculation server participating in calculating is sent to each calculation server of this component of dependence by described by the component relied on; And the component receiving the transmission of each calculation server comprises:

Corresponding component by the component identity relied on according to described, send to and relies on each calculation server of this component by described each calculation server; And receive the component of each calculation server transmission.

4. method as claimed in claim 3,

Described main body-Term Weight matrix comprises: user ID-project main body-Term Weight matrix and user ID-weight equal value matrix;

Described collaborative filtering matrix is the user ID similar matrix of respective user mark-project main body-Term Weight matrix;

Further, the component that described component comprises user ID-project main body-Term Weight matrix enters and user ID-weight equal value matrix component.

5. method as claimed in claim 4, described each calculation server participating in calculating is according to the partitioned matrix data in the collaborative filtering distance matrix of this locality, main body-Term Weight partitioning of matrix matrix data, with component identity in each partitioned matrix data of other all calculation server process, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence comprises:

Each calculation server participating in calculating carries out beta pruning calculating for the partitioned matrix data in described user ID similar matrix;

Each calculation server participating in calculating is according to the partitioned matrix data in the described user ID similar matrix after described beta pruning, and component identity in each partitioned matrix data of other all calculation server process described, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

6. method as claimed in claim 5, described each calculation server participating in calculating carries out beta pruning calculating for the partitioned matrix data in described user ID similar matrix and comprises:

For each dimension of often row or every column matrix data in the partitioned matrix data in user ID similar matrix, the value of each dimension is sorted, at least one dimension retaining every row or often sort forward in row.

7. method as claimed in claim 6, described each calculation server participating in calculating is according to the partitioned matrix data in the described user ID similar matrix after described beta pruning, and component identity in each partitioned matrix data of other all calculation server process described, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence comprises:

Each component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix is carried out row component identity or row component identity transposition by each calculation server participating in calculating;

Align by the row component identity of the result obtained after row component identity transposition with described user ID similar matrix, or align by the result obtained after row component identity transposition with the row component identity of described association user ID similar matrix;

For the dimension that current each row or each row retain, corresponding row component identity or the row component identity retaining dimension of mark;

According to row component identity or the row component identity of described mark, row component identity in the user ID obtained with this locality-project main body-Term Weight matrix and user ID-weight equal value partitioning of matrix matrix data or row component identity compare, and judge local non-existent row component identity or row component identity;

According to the calculation server belonging to the non-existent row component identity in this locality or row component identity, confirm each calculation server that current server relies on, and the component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix in each calculation server relied on.

8. method as claimed in claim 3, described main body-Term Weight matrix comprises: user ID-project main body-Term Weight matrix;

Described collaborative filtering matrix is the project-project similar matrix of respective user mark-project main body-Term Weight matrix;

Further, described component comprises the component of project-project similar matrix.

9. method as claimed in claim 8, described each calculation server participating in calculating is according to the partitioned matrix data in the collaborative filtering distance matrix of this locality, main body-Term Weight partitioning of matrix matrix data, with component identity in each partitioned matrix data of other all calculation server process, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence comprises:

Each calculation server participating in calculating is according to user ID-project main body-Term Weight matrix computations Candidate Recommendation collection;

Each calculation server participating in calculating confirms according to the partitioned matrix data of described Candidate Recommendation collection, project-project similar matrix and user ID-project main body-Term Weight matrix each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

10. method as claimed in claim 9, described each calculation server participating in calculating confirms according to the partitioned matrix data of described Candidate Recommendation collection, project-project similar matrix and user ID-project main body-Term Weight matrix each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence comprises:

Each component identity of described project-project similar matrix is carried out row component identity or row component identity transposition by each calculation server participating in calculating;

Align by the result obtained after row component identity transposition with described user ID-project main body-Term Weight matrix column component identity, or align by the result obtained after row component identity transposition with the row component identity of described association user ID similar matrix;

According to row component identity or the row component identity of described mark, row component identity in the user ID obtained with this locality-project main body-Term Weight matrix and user ID-weight equal value partitioning of matrix matrix data or row component identity compare, and judge local non-existent row component identity or row component identity;

According to the calculation server belonging to the non-existent row component identity in this locality or row component identity, confirm each calculation server that current server relies on, and the component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix in each calculation server relied on.

11. 1 kinds of collaborative filtering disposal systems, comprising:

Request receiving module and at least two calculation servers;

Described request receiver module, is suitable for receiving the request recommending at least one project for main body, starts at least two calculation servers according to described request;

In at least two calculation servers, each described calculation server comprises:

Partitioned matrix data acquisition module, is suitable for each calculation server participating in calculating and obtains various partitioned matrix data; Described each partitioned matrix data comprise main body-Term Weight partitioning of matrix matrix data, collaborative filtering partitioning of matrix matrix data;

Rely on calculation server and confirm module, be suitable for each calculation server participating in calculating according to the main body-Term Weight matrix of this locality and collaborative filtering matrix, component identity with the main body-Term Weight matrix of other calculation servers, confirm each server that current calculation server relies on, and the component of each partitioned matrix data in each calculation server of described dependence; Wherein, described component identity comprises row component identity and row component identity;

Component sending/receiving module, is suitable for each calculation server participating in calculating and is sent to each calculation server of this component of dependence by described by the component relied on; And receive the component of each calculation server transmission;

Recommended project computing module, be suitable for each calculation server participating in calculating for each main body in the main body-Term Weight partitioning of matrix matrix data of this locality, main body-Term Weight partitioning of matrix the matrix data of utilization this locality, the corresponding relation between local collaborative filtering partitioning of matrix matrix data and the component received, give described main body by least one project recommendation.

12. systems as claimed in claim 11, described dependence calculation server confirms that module comprises:

Component identity acquisition module, is suitable for each calculation server participating in calculating and obtains component identity in each partitioned matrix data of other all calculation server process;

First reliance server confirms module, be suitable for each calculation server participating in calculating according to the partitioned matrix data in described collaborative filtering distance matrix, and/or main body-Term Weight partitioning of matrix matrix data, with component identity in each partitioned matrix data of other all calculation server process, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

13. systems as claimed in claim 12, also comprise:

Mark sending module, is suitable for each calculation server participating in calculating by the component identity of each partitioned matrix data in relied on each calculation server, sends to each calculation server that current calculation server relies on;

Further, described component sending/receiving module comprises:

First component sending/receiving module, is suitable for each calculation server participating in calculating according to described by the component identity relied on, is sent to by corresponding component and rely on this component each calculation server; And receive the component of each calculation server transmission.

14. systems as claimed in claim 13,

Described main body-Term Weight matrix comprises: user ID-project main body-Term Weight matrix and user ID-weight equal value matrix;

Described collaborative filtering matrix is the user ID similar matrix of respective user mark-project main body-Term Weight matrix;

Further, the component that described component comprises user ID-project main body-Term Weight matrix enters and user ID-weight equal value matrix component.

15. systems as claimed in claim 14, described first reliance server confirms that module comprises:

Beta pruning module, is suitable for each calculation server participating in calculating and carries out beta pruning calculating for the partitioned matrix data in described user ID similar matrix;

Second reliance server confirms module, be suitable for each calculation server participating in calculating according to the partitioned matrix data in the described user ID similar matrix after described beta pruning, and component identity in each partitioned matrix data of other all calculation server process described, confirm each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

16. systems as claimed in claim 15, described beta pruning module comprises:

First beta pruning module, is suitable for, for each dimension of often row or every column matrix data in the partitioned matrix data in user ID similar matrix, the value of each dimension being sorted, at least one dimension retaining every row or often sort forward in row.

17. systems as claimed in claim 16, described second reliance server confirms that module comprises:

First row/column component identity transpose modules, is suitable for each calculation server participating in calculating and each component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix is carried out row component identity or row component identity transposition;

First row/column component identity alignment module, be suitable for aliging by the row component identity of the result obtained after row component identity transposition with described user ID similar matrix, or align by the result obtained after row component identity transposition with the row component identity of described association user ID similar matrix;

First retains module, is suitable for the dimension retained for current each row or each row, corresponding row component identity or the row component identity retaining dimension of mark;

First judge module, be suitable for the row component identity according to described mark or row component identity, row component identity in the user ID obtained with this locality-project main body-Term Weight matrix and user ID-weight equal value partitioning of matrix matrix data or row component identity compare, and judge local non-existent row component identity or row component identity;

3rd reliance server confirms module, be suitable for the calculation server belonging to the non-existent row component identity in this locality or row component identity, confirm each calculation server that current server relies on, and the component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix in each calculation server relied on.

18. systems as claimed in claim 13, is characterized in that,

Described main body-Term Weight matrix comprises: user ID-project main body-Term Weight matrix;

Described collaborative filtering matrix is the project-project similar matrix of respective user mark-project main body-Term Weight matrix;

Further, described component comprises the component of project-project similar matrix.

19. systems as claimed in claim 18, described reliance server confirms that module comprises:

First Candidate Recommendation collection computing module, is suitable for each calculation server participating in calculating according to user ID-project main body-Term Weight matrix computations Candidate Recommendation collection;

4th reliance server confirms module, be suitable for each calculation server participating in calculating and confirm according to the partitioned matrix data of described Candidate Recommendation collection, project-project similar matrix and user ID-project main body-Term Weight matrix each calculation server that current calculation server relies on, and the component identity of each partitioned matrix data in each calculation server of described dependence.

20. systems as claimed in claim 19, described 4th reliance server confirms that module comprises: the second row/column component identity transpose modules, is suitable for each calculation server participating in calculating and each component identity of described project-project similar matrix is carried out row component identity or row component identity transposition; Second row/column component identity alignment module, be suitable for aliging by the result obtained after row component identity transposition with described user ID-project main body-Term Weight matrix column component identity, or align by the result obtained after row component identity transposition with the row component identity of described association user ID similar matrix; Second judge module, be suitable for the row component identity according to described mark or row component identity, row component identity in the user ID obtained with this locality-project main body-Term Weight matrix and user ID-weight equal value partitioning of matrix matrix data or row component identity compare, and judge local non-existent row component identity or row component identity; 5th reliance server confirms module, be suitable for the calculation server belonging to the non-existent row component identity in this locality or row component identity, confirm each calculation server that current server relies on, and the component identity of user ID-project main body-Term Weight matrix and user ID-weight equal value matrix in each calculation server relied on.