JP4934058B2 - Co-clustering apparatus, co-clustering method, co-clustering program, and recording medium recording the program - Google Patents

Description

  The present invention is based on a non-parametric Bayes model (Dirichlet process mixture model) and co-clustering (selecting user and item selection information (such as purchase history data) indicating whether or not each user has selected each item. It is related to the technique of clustering both items simultaneously: Co-clustering.

  In recent years, the sophistication and diversification of information has progressed, and a technique for efficiently extracting necessary information from a vast amount of information is required. Under such circumstances, for example, as the Internet has become popular, products can be easily purchased from all over the world, and user preferences have diversified. It is becoming increasingly important to understand.

  As one approach for grasping user needs, there is user clustering (segmentation) in which a user group is divided into several user groups having similar purchase histories based on item purchase histories (purchase history data) of a plurality of users. Similarly to user clustering, item clustering is also considered in which item groups are divided into item groups that are easily purchased by the same user. In the case of user clustering, finding the common points of users belonging to the same user group can be used to grasp the needs and preferences of the users. In the case of item clustering, finding common points of items belonging to the same item group can be used for the development of new products.

  Here, not only clustering regarding either a user or an item but also co-clustering is considered. In the co-clustering, when one of the user and the item is clustered, a user / item block (hereinafter, also simply referred to as “block”) determined from the user class and the item class while mutually using the other clustering result is obtained. It is an approach that seeks to obtain more useful information that leads to understanding consumer behavior.

FIG. 10 is a diagram illustrating an example of co-clustering of purchase history data. In FIG. 10, now consider purchase history data regarding whether or not N users have purchased each of the M items. The purchase history data so far is represented by a matrix R as shown in the left diagram of FIG. That is, the (i, j) element R _{i, j} of the N × M matrix R is that the user i (i = 1, 2,..., N) has purchased the item j (j = 1, 2,..., M). It is assumed that “1” is taken when the product is purchased and “0” is taken when the product has not been purchased.

  Here, the right side of FIG. 10 is a result of co-clustering the user and the item. In the right figure, users are sorted so that users belonging to the same user class and items belonging to the same item class are adjacent to each other. By co-clustering in this way, it can be seen that the “1” and “0” portions are more locally concentrated and “blocked”. Note that sorting is always performed in units of users or items.

  By clustering with the item (or user) class added to the constraints when clustering users (or items), the “user / item block” defined from the user class and item class is obtained, and the obtained user -It is possible to grasp the characteristics of the user class and item class from the item block. This method is expected to provide more detailed information than the method of clustering users and items individually.

One method for realizing co-clustering is the Infinite Relational Model (IRM) proposed by Kemp et al. (See Non-Patent Document 1).
In the IRM, first, a plausible data generation process (generation process) for generating given data is modeled. That is, a hypothesis is made that “the given data was generated by such a generation process”. Next, under this hypothesis, a model that best explains the given data is searched.

  Here, with reference to FIG. 11, a data generation process assumed by the IRM will be described using purchase history data. FIG. 11 is an explanatory diagram of an IRM data generation process. As in the case of FIG. 10, “1” is assumed when the user i purchases (has) the item j, and “0” is assumed when the user i has not purchased (does not).

(IRM process 1)
[1-1] The entire user is divided into a plurality of user classes. That is, first, a set of a plurality of user classes (user class set) is generated, and each user selects a user class (one) k and belongs (belongs to) there. Note that the number of user classes K is not fixed but varies, that is, it is determined according to data.
[1-2] Divide the entire item into a plurality of item classes. That is, first, a set of item classes (item class set) is generated, and each item selects an item class (one) s and belongs (belongs to) there. The item class number S is not fixed but varies, that is, it is determined according to the data.

(IRM process 2)
[2-1] A purchase probability specific to a block for a set of any one user class and one item class (ie, for any “user / item block ks”). One real value in the range of “0 to 1” is assigned.
[2-2] It is assumed that a user belonging to a specific user class purchases an item belonging to a specific item class according to the purchase probability assigned to the user / item block.

The given purchase history data is considered to have been generated based on a probability model representing the process, and a model that best explains the given purchase history data is learned, that is, from the given data. Perform model learning.
For example, in the generation process, a user and an item are described as “belonging to (a) class”, but a specific class assignment (ie, clustering) is not described. For specific class assignments, there should be the most appropriate assignment method (which best describes the data) that varies from data to data. This needs to be searched, and doing so will co-cluster the data.

  In general, since there are a large number of user class and item class assignment patterns, it is difficult to obtain an optimal clustering result at a time. Therefore, clustering is performed using the sampling described below. In sampling, first, a random clustering result is used as an initial value, and the clustering result is sequentially improved so that the posterior probability as an objective function is increased.

  Here, the posterior probability is a kind of conditional probability, and is a probability related to a specific variable under a condition that takes into account specific information. The prior probability is a probability related to a specific variable under the condition where there is no specific information. For example, according to Bayes' theorem, the posterior probability is obtained by multiplying the prior probability by the likelihood (function). The posterior probability can be a measure of the goodness of the model learning results.

  Continuing the description of the IRM, specifically, first, users and items are randomly clustered as initial values. Next, each of the user belonging user class and the item belonging item class is sequentially updated so that the posterior probability (formula (5) described later) of the given purchase history data is increased. When the posterior probability, which is the evaluation value, is no longer changed (improved), the attribution class update is stopped.

Hereinafter, the probability model assumed by the IRM and the evaluation formula of the co-clustering result will be described. First, the following are the quantities observed as data.
The number of users is N and the number of items is M.
The matrix R represents purchase history data, and the (i, j) element R _{i, j} of the N × M matrix R represents the user i (i = 1, 2,..., N) as the item j (j = 1, 2, .., M) is “1” when purchased, and “0” when not purchased.

Once these are obtained, the following probabilistic model is assumed.
(Advance probability of user class assignment)
Z = {z ₁ , z ₂ ,..., Z _N } represents the user class assignment, and z _i = k means that the user class to which user i belongs is k. Therefore, if the number of user classes is K, z _i ε {1, 2,..., K} (i = 1, 2,..., N). At this time, if _nk is the number of users belonging to the user class k (the number of i for which z _i = k: user class size), Σ ^K _{k = 1} n _k = N (“Σ ^K _{k = 1} ” is “ k ”takes a value from“ 1 ”to“ K ”(the same applies hereinafter), and the prior probability of the user class assignment Z to the user is expressed by equation (1) (user class assignment probability equation) ) However, α> 0 is a constant (hyper parameter).

Here, the prior probability (distribution) can be considered as a measure representing the “goodness” of the classification in the state before the purchase history data R is known (that is, the “prior” state of R observation). . As the number of user classes K and the user class size _nk change, the value of P (Z; α) shown in Expression (1) changes. Intuitively, this means that the larger the user class size _{nk and the} smaller the number K of user classes (when α <1), the better the class assignment is considered. Note that the number K of user classes is not fixed in advance, but is sequentially updated (changed) so as to be a numerical value suitable for given data, and finally determined.

  In the state after knowing the purchase history data R, basically, each user / item block has “1” or “0”, which is “1” or “0” as a result of co-clustering. Ideally it should be a clear block. However, if we aim only at that (optimizing only that as an objective function), we will try to make as small a block as possible. This is because, as an extreme example, if all the elements of all user / item blocks are made one by one, “1” and “0” are the clearest. However, such a state that “all elements are different blocks” is not desirable because it is not balanced from the viewpoint of “block division”. It is the prior probability P (Z; α) that corrects this. The same applies to P (W; β) described later.

(Priority probability of item class assignment)
Similarly, W = {w ₁ , w ₂ ,..., W _M } represents the class assignment of the item, and w _j = s means that the belonging item class of item j is s. Therefore, if the number of item classes is S, w _j ε {1, 2,..., S} (j = 1, 2,..., M). At this time, if m _s is the number of items belonging to the item class s (the number of j where w _j = s), Σ ^S _{s = 1} m _s = M, and the prior probability of the item class assignment W to the item Is the equation (2) (item class assignment probability equation). Further, the item class number S is not fixed in advance, but is sequentially updated (changed) so as to be a numerical value suitable for given data, and finally determined. β> 0 is a constant (hyper parameter).

ただし、Ｒの各要素はおのおの独立に生成されたものと仮定する。式（３）は、Ｒが与えられた下での、Θの「良さ」（尤度）である。 (Appearance probability of matrix R)
Further, Θ = {θ _{k, s} | k = 1, 2,..., K, s = 1, 2,..., S} is a user item block (k, s) determined from the user class k and the item class s. The purchase probability for each. In other words, the user belonging to the user class k means that an item that belongs to the item class s to buy with a probability of _{θ k, s ((1-} θ k, s) do not buy with a probability of) ( "Bernoulli Called "trial"). Therefore, the appearance probability of R, which is observation data, given Z, W, and Θ is expressed by Equation (3). As described above, the user class number K and the item class number S are not fixed, but are updated sequentially.

However, it is assumed that each element of R is generated independently. Equation (3) is the “goodness” (likelihood) of Θ under the condition that R is given.

(Purchase probability)
theta _{k, s} is a mere "0" to "1" in size rather than being generated by the uniform random number, as are normally produced by the beta distribution, the considered (i.e. prior distribution theta _{k, s} Use beta distribution). Therefore, if the parameter of the beta distribution is γ = {γ ₀ , γ ₁ } _, the prior distribution of θ _{k, s} is expressed by equation (4). Note that Γ (x) is a gamma function.

The above is the probability model assumed in the IRM.
Under this model, the optimum co-clustering result is obtained for the given purchase history data R. With respect to the user class assignment Z and the item class assignment W, which are co-clustering results, the posterior probabilities of Z and W under R are given by Equation (5).
P (Z, W | R; α, β, γ) (5)

  The optimal co-clustering is nothing but to obtain a co-clustering result in which the posterior probability given by the equation (5) is the maximum (the maximum is not the strict maximum but may be calculated). That is, Expression (5) is an evaluation function representing the goodness of application of the co-clustering results Z and W to the purchase history data R, and it can be considered that the larger this value, the better the co-clustering result.

Equation (5) can be calculated as follows. That is, under the assumption of IRM, the probability model (simultaneous distribution of purchase history data) can be expressed as equation (6) using equations (1), (2), (3), and (4).
P (R, Θ, Z, W; α, β, γ) =
P (R | Θ, Z, W) P (Z; α) P (W; β) P (Θ; γ) (6)

Here, the conditional distribution under the purchase history data R can be expressed as shown in Equation (7).
P (Z, W, Θ | R; α, β, γ) (7)

From Bayes' theorem, equation (7) can be expressed as equation (8).

Since the denominator P (R; α, β, γ) on the right side of Equation (8) is a constant, the numerator term is considered. That is, it can be expressed as equations (9) and (10).

Here, since the beta distribution, which is a conjugate prior distribution of Bernoulli distribution, is used as the prior distribution of θ _{k, s} (∈Θ), Θ can be integrated and eliminated as expressed in equations (11) and (12). . That is, if both the left side of the formula (9) and the formula (10) are integrated with respect to Θ, and then the integration of the formula (11) is executed, Θ disappears as shown in the formula (12).

  The left side of Equation (11) is nothing but P (Z, W | R; α, β, γ) in Equation (5), and this can be calculated by Equation (12). Each term of Formula (12) can be calculated from Formulas (1), (2), and (3), respectively.

The feature of IRM is that the number of classes K and S are automatically calculated from the given data in the framework of statistical learning without giving the number of classes that most of the conventional (co) clustering methods require in advance. It is in the point that can be decided. Usually, the number of classes in actual data such as purchase history data is unknown in advance, and there is no need to search for the number of classes based on an evaluation scale for the number of classes unlike other conventional methods.
Kemp, C., Tenenbaum, JB, Griffiths, TL, Yamada, T. & Ueda, N., “Learning systems of concepts with an infinite relational model.” In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-06) , pages 381-388, June 2006.

However, when IRM is applied to purchase history data, the following two points are problematic.
(Problem 1)
The purchase history data R is sparse. That, R _i, in comparison with the j = 1 (buy) become elements, even though R _i, j = 0 (non purchased) and a number of elements is overwhelmingly large, IRM is R _i, j = Since the model handles 1 and R _{i, j} = 0 on an equal basis, it is difficult to handle purchase history data (results are not accurate).

(Problem 2)
In the purchase history data R, an element with R _{i, j} = 0 does not necessarily mean “not purchased”, but rather should be interpreted as “unknown whether or not to purchase”. For example, in general, the user thinks that there is a possibility that an item that is not yet purchased will be purchased in the future (where R _{i, j} = 0 becomes R _{i, j} = 1). Is natural. That is, the purchase history data R can be regarded as data having many missing values. However, since IRM is not a model devised for data with missing values, such a situation cannot be handled well (results are not accurate).

  In summary, it can be said that the IRM which is the prior art is not necessarily a model suitable for handling purchase history data.

Accordingly, the present invention has been made to solve the above-described problem, and data including missing values such as purchase history data (in the purchase history data R, a location where R _{i , j} = 0 is R _{i. , j} = data that may not be 0) based on a more suitable generation process (probability model), it is an object to co-cluster users and items.

  In order to solve the above-mentioned problem, the present invention performs clustering by treating item selection information indicating whether or not each user has selected each item or frequency of selection as matrix information related to the user and the item. Classifying each of the users in the matrix information into a plurality of user classes based on a predetermined user class allocation probability formula, and classifying each of the items in the matrix information based on a predetermined item class allocation probability formula For each user / item block uniquely identified by a combination of the user class and the item class, a user of the user class selects an item of the item class on the probability model. Preliminary selection probability that is probability And maximizing the posterior probability of the matrix information calculated from the predetermined user class allocation probability formula, the predetermined item class allocation probability formula, and the predetermined item selection probability formula. In the co-clustering apparatus for clustering the matrix information in both the user unit and the item unit, a predetermined item class designation probability for each item class is given to each user class, and each item described above A predetermined user class designation probability for each user class is given to the class, and the predetermined item selection probability formula is obtained by calculating the predetermined item class designation probability and the predetermined user class designation probability for each user / item block. The provisional selection probability as a multiplication value or a value based on the multiplication value of A storage unit that stores the item selection information and the matrix information, and an input of the latest matrix information, and a user class to which one or more of the users in the matrix information belong, A user class update unit that updates based on a predetermined user class assignment probability formula and the provisional selection probability, and receives the latest matrix information input, and an item class to which one or more items in the matrix information belong An item class updating unit that updates based on the predetermined item class allocation probability formula and the provisional selection probability, and a co-clustering result evaluation unit that calculates a posterior probability of the matrix information based on the latest matrix information; .

  According to this invention, the predetermined item selection probability formula calculates the provisional selection probability as a product of the predetermined item class specification probability and the predetermined user class specification probability or a value based on the multiplication value for each user / item block. A user class is updated based on a predetermined user class allocation probability formula and a provisional selection probability, an item class is updated based on a predetermined item class allocation probability expression and a provisional selection probability, and the latest matrix By calculating the posterior probability of matrix information based on the information, co-clustering users and items based on a more suitable generation process (probability model) to data containing missing values such as purchase history data Can do.

  In addition, the present invention repeats each step by the user class update unit, the item class update unit, and the co-clustering result evaluation unit until a predetermined condition is satisfied, and when the predetermined condition is satisfied, the latest matrix information It is desirable to further include a co-clustering end determination unit that stores a result of co-clustering related to the user and the item based on the item in the storage unit.

  According to this invention, when the predetermined condition is satisfied, the result of the co-clustering regarding the user and the item based on the latest matrix information is stored in the storage unit, so that the co-clustering can be terminated at an appropriate timing.

  Further, according to the present invention, the user class update unit selects one of the users to be updated from the latest matrix information, and each of the existing and new user classes to which the selected user is transferred. It is desirable to calculate a migration probability, determine the user class to which the user is to be migrated according to the respective migration probability, and update information on the user class based on the determination.

  According to this invention, it is possible to avoid a situation in which co-clustering falls into a deadlock (inappropriate stalemate state) by determining a user class to which a user is to be migrated according to the migration probability.

  Further, according to the present invention, the item class update unit selects one of the items to be updated from the latest matrix information and transfers each of the existing and new item classes to which the selected item is to be transferred. It is desirable to calculate a transfer probability, determine the item class to which the item is to be transferred according to the respective transfer probabilities, and update information on the item class based on the determination.

  According to this invention, it is possible to avoid a situation in which co-clustering falls into a deadlock (inappropriate stalemate state) by determining an item class to which an item is transferred according to the transfer probability.

  The co-clustering program according to the present invention is characterized by causing a computer constituting the co-clustering apparatus to be executed. With such a configuration, a computer in which this program is installed can realize each function based on this program.

  A computer-readable recording medium according to the present invention is characterized in that the above-described co-clustering program is recorded. With such a configuration, a computer equipped with the recording medium can realize each function based on a program recorded on the recording medium.

  According to the present invention, users and items can be co-clustered on data including missing values such as purchase history data based on a more suitable generation process (probability model).

  The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described in detail below with reference to the drawings (refer to drawings other than the referenced drawings as appropriate). First, to help understanding, the data generation process of this embodiment will be described, and then the co-clustering apparatus will be described.

<Data generation process>
In this embodiment, a data generation process different from the data generation process assumed by the IRM is assumed. FIG. 9 is an explanatory diagram of a data generation process assumed in the present embodiment.

(Process 1 of this embodiment)
[1-1] The entire user is divided into a plurality of user classes. That is, first, a set of a plurality of user classes (user class set) is generated, and each user selects a user class (one) k and belongs (belongs to) there.
[1-2] Divide the entire item into a plurality of item classes. That is, first, a set of item classes (item class set) is generated, and each item selects an item class (one) s and belongs (belongs to) there.
The process 1 of this embodiment is exactly the same as the process 1 of the IRM described above.

(Process 2 of this embodiment)
[2-1] The probability (item class designation probability) of which class among the item classes is easily selected (designated) is assigned to each user class. That is, one multinomial distribution on the item class is assigned to each user class k. This corresponds to assigning _s non-negative real numbers θ _{k, s} (≧ 0) such that Σ ^S _{s = 1} θ _{k, s} = 1, _where S is the number of item classes. Here, θ _{k, s} is the probability of selecting (designating) the item class s for users belonging to the user class k, and corresponds to the probability of being noticed as s when compared to a dice.

[2-2] Similarly, each item class is assigned a probability (user class designation probability) that it is easy to select (designate) which user class among the user classes. That is, one multinomial distribution on the user class is assigned to each item class s. This is equivalent to allocating K non-negative real numbers φ _{s, k} (≧ 0) such that Σ ^K _{k = 1} φ _{s, k} = 1 _{, where} K is the number of user classes. φ _{s, k} is the probability of selecting (designating) the user class k for users belonging to the item class s.

[2-3] Each user selects (specifies) an item class based on the item class designation probability of the user class to which the user belongs, and each item has a user class designation probability of the item class to which the user belongs. Select (specify) the user class based on this. As a result, when the two match, the corresponding user is considered (deemed) to purchase the corresponding item.
The process 2 of the present embodiment is greatly different from the process 2 of the IRM described above.

  As in the case of IRM, in this embodiment, the given purchase history data (item selection information) is regarded as being generated based on a probability model representing the process, and the data is co-clustered. Specifically, as in the case of IRM, a co-clustering result is obtained by sequentially updating each of the user belonging user class and the item belonging item class from a state in which the user and the item are randomly clustered. Here, similarly to the IRM, in this embodiment, the number of classes can be automatically determined from the given data in the framework of statistical learning.

Next, the probability model assumed by this embodiment and the evaluation formula of the co-clustering result will be described. As in the case of IRM, first, the following are the quantities observed as data.
The number of users is N and the number of items is M.
The matrix R represents purchase history data, and the (i, j) element R _{i, j} of the N × M matrix R represents the user i (i = 1, 2,..., N) as the item j (j = 1, 2, .., M) is “1” when purchased, and “0” when not purchased.

Once these are obtained, the following probabilistic model is assumed.
(Prior probability of user class assignment, prior probability of item class assignment)
P (Z; α) and P (W; β) are exactly the same as in the case of IRM, and formulas (1) and (2) are used as they are.
(Appearance probability of matrix R)
The appearance probability of R, which is observation data, given Z, W, Θ, and Φ can be expressed as Equation (13).

This equation (13) is the “goodness” (likelihood) of Θ and Φ under the condition that R is given. This equation (13) is one of the features of this embodiment, and is greatly different from the corresponding equation (3) of IRM. Here, the process of assuming that the user class z _i and the item class w _j purchase only when the same item class w _j and user class z _i are selected (designated) is expressed by the following equation (13) (item selection probability formula ) In (θ _{zi, wj φwj, zi} ) ^{Ri, j} .

In this “(θ _{zi, wj φwj, zi} ) ^{Ri, j} ”, “θ _{zi, wj} ” is the probability that the user belonging to the user class z _i will select (specify) the item class w _j (item class designation probability) ). “Φ _{wj, zi} ” is a probability (user class designation probability) that an item belonging to the item class w _j selects (designates) the user class z _i . Therefore, in this model, “θ _{zi, wj} φ _{wj, zi} ” is a probability that a user belonging to the user class z _i purchases an item belonging to the item class w _j (provisional selection probability).

“R _{i, j} ” is a numerical value “1” when “purchase” and “0” when “not purchase”. Therefore, the value of “(θ _{zi, wj φwj, zi} ) ^{Ri, j} ” becomes “(θ _{zi, wj φwj, zi} )” (“θ _{zi, wj} ” and “φ _{wj , zi} ”) is left as it is and becomes“ 1 ”when“ not purchased ”(because the zero power of a numerical value other than“ 0 ”is“ 1 ”. The 0th power may be treated as “1” for convenience). That is, Expression (13) obtained by taking all products of “(θ _{zi, wj φwj, zi} ) ^{Ri, j} ” is the appearance probability of the matrix R as the observation data in this model.

  The provisional selection probability may be a value calculated based on the multiplication value, such as a power of the multiplication value or a power root, instead of the multiplication value itself.

By calculating the appearance probability of the matrix R in this way, the number of elements with R _{i, j} = 0 (not purchased) is overwhelmingly larger than the elements with R _{i, j} = 1 (purchased). In this case, a data generation process suitable for the case of purchase history data including missing values can be realized.

ただし、以下、簡単のためγ_ｋ＝γとする。 (Probability that a user belonging to user class k selects (specifies) item class s)
Θ = {θ _{k, s} | k = 1, 2,..., K, s = 1, 2,..., S} (where θ _{k, s} ≧ 0, Σ ^S _{s = 1} θ _{k, s} = 1) As a prior probability (distribution), a Dirichlet distribution as shown in Equation (14) is used.

However, for the sake of simplicity, it is assumed that γ _k = γ.

ただし、以下、簡単のためη_ｓ＝ηとする。 (Probability that an item belonging to item class s selects (designates) user class k)
_{Similarly, Φ = {φ s, k} | s = 1,2, ..., S, k = 1,2, ..., K} ( _{however, φ s, k ≧ 0,} Σ K k = 1 φ s, k As a prior distribution of = 1), a Dirichlet distribution as in Expression (15) is used.

However, for simplicity, η _s = η is assumed below.

  Expressions (14) and (15) described above correspond to Expression (4) in IRM. The above is the probability model assumed in the present embodiment.

ここで、Θ＝｛θ_ｋ,ｓ｜ｋ＝１,２,…,Ｋ, ｓ＝１,２,…,Ｓ｝,Φ＝｛φ_ｓ,ｋ｜ｓ＝１,２,…,Ｓ, ｋ＝１,２,…,Ｋ｝はそれぞれ多項分布のパラメータを表し、また、a、β、γ、ηはハイパーパラメータ(定数)を表す。 As in the case of the IRM, let us consider obtaining a co-clustering result that maximizes the expression (5) that is the posterior probability of Z and W. In order to calculate this, first, the probability model (simultaneous distribution of purchase history data) in the present embodiment can be expressed as Equations (16) and (17).

Here, Θ = {θ _{k, s} | k = 1, 2,..., K, s = 1, 2,..., S}, Φ = {φ _{s, k} | s = 1, 2,. k = 1, 2,..., K} each represent a parameter of a multinomial distribution, and a, β, γ, and η represent hyperparameters (constants).

By using the Dirichlet distribution, which is a conjugate prior distribution of the multinomial distribution, as the prior distribution of Θ and Φ, in practice, Θ and Φ can be integrated and eliminated as in equations (18) and (19). That is, if the left side of Expression (16) and both sides of Expression (17) are integrated with respect to Θ and Φ, and then the integration of Expression (18) is performed, Θ and Φ disappear as shown in Expression (19). .

Therefore, this integral can be solved analytically and there is no need to explicitly consider Θ and Φ. When the integration is actually executed and specifically substituted, the posterior probabilities of Z and W are as shown in equations (20) and (21).

ただし、δ(ｘ)はｘが真の場合に「１」、それ以外に「０」となる関数(デルタ関数)とする。 Note that l (k, s) represents the number of times a user belonging to the user class k has purchased an item belonging to the item class s. That is, the number of times is as shown in Expression (22).

However, δ (x) is a function (delta function) that becomes “1” when x is true and “0” otherwise.

  In the present embodiment, co-clustering is equivalent to obtaining Z and W that maximize Equation (21).

<Co-clustering device>
Next, the co-clustering apparatus of this embodiment will be described. In the present embodiment, a sampling algorithm is used. FIG. 1 is a functional block diagram schematically showing the configuration of the co-clustering apparatus according to this embodiment. The co-clustering device 100 is a computer device that includes, for example, a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), and an input / output interface. As shown in FIG. 1, the co-clustering apparatus 100 includes a calculation unit 1, a storage unit 2 (storage unit), an input unit 3 such as a keyboard and a mouse, and an output unit 4 such as a liquid crystal display, which are bus lines 5. Connected with.

The computing means 1 includes a preprocessing unit 11, an initialization unit 12, a user class update unit 13, an item class update unit 14, a co-clustering result evaluation unit 15, a co-clustering end determination unit 16, and a memory 17 that is an operation area. I have.
The storage unit 2 includes a purchase history DB 21 that stores purchase history data (matrix information), a co-clustering result storage unit 22 that stores (stores) a co-clustering result, and a program 23.

The program 23 includes functional programs of preprocessing 231, initialization 232, user class update 233, item class update 234, co-clustering result evaluation 235, and co-clustering end determination 236.
The preprocessing unit 11, initialization unit 12, user class update unit 13, item class update unit 14, co-clustering result evaluation unit 15, and co-clustering end determination unit 16 of the computing unit 1 are respectively pre-process 231, program 23, This is realized by reading the function programs of initialization 232, user class update 233, item class update 234, co-clustering result evaluation 235 and co-clustering end determination 236 into the memory 17 and developing them.

  Here, the flow of processing of the co-clustering apparatus 100 will be described with reference to FIG. 2, and the operation of each part of the computing means 1 will be described in detail. FIG. 2 is a flowchart showing a processing flow of the co-clustering apparatus 100.

First, the preprocessing unit 11 performs preprocessing (step S1). Specifically, the preprocessing unit 11 creates a matrix R from the purchase history data stored in the purchase history DB 21. Here, R _{i, j} representing the (i, j) element of R is "1" when the user i had purchased the item j, and "0" when not purchased. The number of users is N, and the number of items is M.

After step S1, the initialization unit 12 performs initialization (step S2). Specifically, the initialization unit 12 sets the user's temporary attribution user class Z = {z _i | i = 1, 2,..., N} and the item's temporary attribution item class W = {w _j | j = 1, 2,..., M} are appropriately determined. Thereby, the temporary user class number K and the temporary item class number S are also determined. Here, for example, a method of randomly assigning the belonging class or a method of assigning each data to different classes (in this case, the number of user classes is N and the number of item classes is M) is used.

  Subsequently, the user class update unit 13 performs user class update (step S3). Step S3 will be described with reference to FIG. FIG. 3 is a flowchart obtained by subdividing the processing in step S3.

First, the user class update unit 13 randomly selects one user whose user class is to be updated, and sets the selected user's index to i (step S31: update target user selection). Next, a probability that the user i selects the user class k ^* (ε {1, 2,..., K, K + 1}) is calculated (step S32: user class belonging probability calculation), and the user i is calculated based on the calculated probability. Is determined (step S33: determination of belonging user class). In this way, by determining the user migration destination user class based on the (migration) probability, it is possible to avoid a situation in which co-clustering falls into a deadlock (an inappropriate stalemate state).

Finally, to update the z _i by determining the assigned user class (step S34: User Class Update). When an empty user class is generated by this user class update, the user class is deleted, and the number of user classes is reduced. Conversely, when a new user class is generated by this user class update, the number of user classes increases accordingly.

Here, step S32 will be described in detail. The probability that the user i selects the user class k ^* (user class allocation probability formula) is represented by the formula (23). However, k ^* = 1, 2,..., K, K + 1.
P (z _i = k ^* | R, Z ₋₁ , W; α, β, γ, η) (23)
Z ₋₁ = {z ₁ , z ₂ ,..., Z _i−2 , z _i−1 , z _{i + 1} , z _{i + 2} ,..., Z _N−1 , z _N } (Z excluding z _i ), Expression (23) can be expressed as Expressions (24), (25), and (26).

Note that k ^* ∈ {1,2, ..., K, K + 1}, that is, calculate the probability of selecting the new K + 1 class in addition to the existing K classes. It is. When k ^* is an existing class, it can be expressed as in equation (27), and when k ^* is a new class, it can be expressed as in equation (28).

ｋ^＊が新規クラスである場合：

If k ^* is an existing class:

If k ^* is a new class:

Note that n− _{i, k} represents the number of users belonging to the user class k when the user i is removed from the purchase history data, and l (k, s) is the user as defined by the equation (22). This represents the number of times a user belonging to class k has purchased an item belonging to item class s. L _−i (k, s) is l (k, s) when user i is removed from the purchase history data, and l _{+ i} (k, s) is l (when user i is assigned to user class k). k, s).

  Returning to FIG. 2 and continuing the description, after step S3, the item class updating unit 14 updates the item class (step S4). Step S4 will be described with reference to FIG. FIG. 4 is a flowchart obtained by subdividing the processing in step S4.

First, the item class updating unit 14 randomly selects one item for which the item class is to be updated, and sets the index of the selected item to j (step S41: update target item selection). Next, the probability that the item j selects the item class s ^* (ε {1, 2,..., S, S + 1}) is calculated (step S42: item class attribution probability calculation), and the item j is based on the calculated probability. Is determined (step S43: determination of belonging item class). Thus, by determining the item class to which the item is to be transferred based on the (migration) probability, it is possible to avoid a situation in which co-clustering falls into a deadlock (an inappropriate stalemate state).

Finally, w _j is updated with the determined belonging item class (step S44: item class update). When an empty item class is generated by this item class update, the item class is deleted, and the number of item classes is reduced. Conversely, when a new item class is generated by this item class update, the number of item classes increases accordingly.

Here, step S42 will be described in detail. The probability that the item j selects the item class s ^* (item class assignment probability equation) is expressed as equation (29). However, s ^* = 1, 2,..., S, S + 1.
P (w _j = s ^* | R, Z, W _−j ; α, β, γ, η) (29)

_Further, W -j = minus the _{w j} from _{{w 1, w 2, ...} , w j-2, w j-1, w j + 1, w j + 2, ..., w N-1, w N} (W ) And R _j = {R _1j , R _2j ,..., R _Nj }, Expression (29) can be expressed as Expressions (30), (31), and (32).

Here, as in the case of the user class update unit, s ^* ε {1, 2,..., S, S + 1}, and the probability of selecting the new S + 1th class in addition to the existing S classes is also calculated. To do. When s ^* is an existing class, it can be expressed as in equation (33), and when s ^* is a new class, it can be expressed as in equation (34).

ｓ^＊が新規クラスである場合：

If s ^* is an existing class:

If s ^* is a new class:

Here, m _{−j, s} represents the number of items belonging to the item class s when the item j is removed from the purchase history data, and l _−j (k, s) is when the item j is removed from the purchase history data. L (k, s) and l _{+ j} (k, s) represent l (k, s) when item j is assigned to class k.

Returning to FIG. 2 and continuing the description, after step S4, the co-clustering result evaluation unit 15 performs co-clustering result evaluation, that is, evaluates the user clustering result Z and the item clustering result W (step S5). For the evaluation, the posterior probabilities of the user clustering result Z and the item clustering result W are calculated from the equations (35) and (36) and used.
The larger the value calculated by Equations (35) and (36), the better the clustering result.

  After step S5, the co-clustering end determination unit 16 determines whether to end the co-clustering, that is, whether to end the update of the user class and the update of the item class (step S6). For example, the process may be terminated when there is no change in the evaluation value (Z, W posterior probability) of the co-clustering result. Alternatively, the process may be terminated when the number of updates specified in advance is exceeded. Or you may perform this completion | finish determination by those combination and other conditions.

If it is determined in step S6 that “continue (do not end)”, the process returns to step S3.
When it is determined as “end” in step S6, the co-clustering end determination unit 16 stores the co-clustering result (Z, W) in the co-clustering result storage unit 22 (step S7). By this step S6, the co-clustering can be terminated at an appropriate timing.

  As described above, according to the co-clustering apparatus 100 of the present embodiment, based on a generation process (probability model) more suitable for data including missing values such as purchase history data, the user and the Items can be co-clustered well. In particular, the differences from the conventional IRM are summarized as follows.

  In the IRM, both the event that the user purchases the item and the event that the user does not purchase are considered equally, and the user and the item are bi-directionally clustered based on the Bernoulli distribution as expressed by Equation (3).

  On the other hand, in the present embodiment, it is assumed (not assumed) that the user purchases or does not purchase the item, but the user and the item select (designate) each other and purchase the item when they match each other (formula 13). The user and the item select each other according to the multinomial distribution represented by

  In other words, the IRM clusters users and items bidirectionally based on the Bernoulli distribution in the user / item block, whereas this embodiment uses the multinomial distribution on the user class / item class. And bi-directionally cluster items.

  By considering only an event in which the user purchases an item (an event in which the user and the item select (designate) each other), it is not necessary to consider an event in which the user does not purchase the item. As a result, it is possible to deal with a missing portion in the purchase history data, that is, an item portion that is not purchased but is purchased in the future.

  Note that, by creating a co-clustering program to be executed by a computer constituting the co-clustering apparatus 100 and installing the co-clustering program on the computer, the computer can realize each function based on the co-clustering program. The co-clustering program can be recorded on various recording media such as a CD (Compact Disc) and a DVD (Digital Versatile Disc).

As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning.
For example, the purchase history data to which the present invention is applied is “0” indicating the frequency of selection even if each element of the matrix R is not indicated by binary values such as “0” and “1”. It may be represented by an integer greater than or equal to. In this case, each element of the matrix R indicates “the number of purchases” instead of “purchased / not purchased” (“0” indicates that it has not been purchased). Consider the number of purchases.

  In addition to the purchase history data, the present invention provides access history data of a user's Web page (Web page corresponds to an item), document / word data (a document is a user, and a word appearing in the document corresponds to an item) , Paper / author data (the paper is the user, the author who wrote the paper is the item), and other data. In other words, in the claims, “user” does not necessarily indicate “person”, and “item” does not necessarily indicate “thing”, and may be interpreted as other than appropriate. .

Further, when performing co-clustering based on the model of the present embodiment, the above-described description has shown the sampling method, but a method using another learning algorithm such as a variational Bayes algorithm can also be adopted.
Furthermore, in order to obtain co-clustering results Z and W with larger evaluation values (to prevent falling into poor local solutions), the respective class assignments Z and W are probabilistic during user class / item class update. It is also possible to add an operation to change to.

Moreover, you may make it change a some user or item to another class simultaneously.
In addition, specific configurations of hardware and software can be appropriately changed without departing from the gist of the present invention.

<Example>
Next, a comparison between the case where “co-clustering by co-clustering apparatus 100 of the present embodiment” is executed on real data (purchase history data) and the case where “co-clustering by IRM (conventional technology)” is executed is performed. The results will be described.

(Data used)
As actual data, data “MovieLens” composed of a history actually evaluated by a user for a movie was used. The evaluation value takes five values of “1”, “2”, “3”, “4”, and “5”. Then, the movie evaluated by the user is regarded as an item purchased by the user, the movie not evaluated by the user is regarded as an item not purchased by the user, and purchase history data of 943 users and 1682 items is created. did. FIG. 5 is a diagram showing a matrix R of the purchase history data. In FIG. 5, the vertical axis represents the user, the horizontal axis represents the item, the black dot is R _{i, j} = 1 (purchased product), and the white dot is R _{i, j} = 0 (purchase product) (The same applies to FIGS. 6 and 7).

(Implementation method)
Hyperparameter values in each method are as follows.
This embodiment: α = β = 0.1, γ = η = 0.01 (k = 1, 2,..., K, s = 1, 2,..., S)
IRM: α = β = 0.1, γ = η = 0.01 (k = 1, 2,..., K, s = 1, 2,..., S)

  The initialization unit 12 initializes Z and W by randomly dividing the user and the item into 100 classes, respectively. The co-clustering end determination unit 16 continuously evaluates the evaluation values (Z and W posterior probabilities). Co-clustering was terminated when (distribution)) was not updated.

(Implementation results)
Qualitatively compare and evaluate the results of applying the IRM and the method according to the present embodiment to the purchase history data R, respectively. FIG. 6 is a diagram illustrating an application result of the IRM technique. FIG. 7 is a diagram illustrating an application result of the method of the present embodiment.
In FIG. 6 and FIG. 7, users and items are sorted for each user class and item class obtained by co-clustering, and class divisions are represented by solid lines. Each class is arranged in descending order from the top left.

The number of user classes / number of item classes obtained by each method was 39/54 in the IRM method and 33/57 in the method of this embodiment. It can be seen that by either method, a result that the purchase history (black portion) is densely obtained for each user / item block is obtained. However, in the IRM method, a large user / item block (upper left portion in FIG. 6) is configured by collectively collecting user / item groups having a small purchase history, and R occupies most of the purchase history data. It can be seen that _{i, j} = 0. On the other hand, in the method of the present embodiment, the portion of R _{i, j} = 1 (black) is distributed as shown in FIG. 7 as compared with the IRM method, and R _{i, j} = 0. It can be confirmed that there is not much influence of.

  Here, FIG. 8 is a diagram showing a part of a movie title for each item class obtained by the method of the present embodiment. As shown in FIG. 8, class 6 (item class 6) is a hit product, class 29 (item class 29) is a horror movie, and class 35 (item class 35) is an item class for movies for children and families. It can be seen that a characteristic item class is obtained by the method of the present embodiment.

With the IRM method, clustering results with high accuracy cannot be obtained so far. To explain again, for example, as shown in FIG. 6, the IRM method is strongly influenced by R _{i, j} = 0 (not purchased), and a large white item class 1 area is formed at the left end. The area of the item class 1 includes various movie titles extending over a plurality of genres (specific title names are omitted). The most serious problem in the IRM method is the generation of a large white item class area at the left end. Since this area is a gathering area for movies with few accesses, movies of various genres are mixed. On the other hand, as shown in FIG. 7, in the method of the present embodiment, such a large white item class region does not occur.

It is a functional block diagram which shows typically the structure of the co-clustering apparatus which concerns on this embodiment. It is a flowchart which shows the flow of a process of a co-clustering apparatus. It is the flowchart which subdivided the process of step S3. It is the flowchart which subdivided the process of step S4. It is a figure which shows the matrix R of real data (purchase history data). It is a figure which shows the application result of the method of IRM. It is a figure which shows the application result of the method of this embodiment. It is a figure which shows a part of movie title for every item class obtained by the method of this embodiment. It is explanatory drawing of the data generation process assumed in this embodiment. It is a figure which shows the example of co-clustering of purchase history data. It is explanatory drawing of the data generation process of IRM.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Calculation means 2 Storage means 11 Pre-processing part 12 Initialization part 13 User class update part 14 Item class update part 15 Co-clustering result evaluation part 16 Co-clustering end determination part 21 Purchasing history DB
22 Co-clustering result storage unit 23 Program 100 Co-clustering device

Claims (10)

In order to cluster item selection information indicating whether each user has selected each item or the frequency of selection, as matrix information related to the user and the item,
Based on a predetermined user class allocation probability formula, classify each of the users in the matrix information into a plurality of user classes,
Based on a predetermined item class allocation probability formula, classify each of the items in the matrix information into a plurality of item classes,
For each user / item block uniquely specified by the combination of the user class and the item class, a provisional selection probability that is a provisional probability that a user of the user class selects an item of the item class on the probability model Based on a predetermined item selection probability formula,
The matrix information is calculated by maximizing the posterior probability of the matrix information calculated from the predetermined user class allocation probability formula, the predetermined item class allocation probability formula, and the predetermined item selection probability formula. A co-clustering device that performs clustering in both user units and item units,
A predetermined item class designation probability for each item class is given to each user class, a predetermined user class designation probability for each user class is given to each item class, and the predetermined item selection probability formula is A formula for calculating the provisional selection probability as a multiplication value or a value based on the multiplication value of the predetermined item class designation probability and the predetermined user class designation probability for each user / item block,
A storage unit for storing the item selection information and the matrix information;
A user class update unit that receives the latest input of the matrix information and updates a user class to which one or more of the users belong in the matrix information based on the predetermined user class allocation probability formula and the provisional selection probability When,
An item class update unit that accepts the latest input of the matrix information and updates an item class to which one or more of the items in the matrix information belong based on the predetermined item class allocation probability formula and the provisional selection probability When,
Based on the latest matrix information, a co-clustering result evaluation unit that calculates the posterior probability of the matrix information;
A co-clustering apparatus comprising: Until the predetermined condition is satisfied, the processing by the user class update unit, the item class update unit, and the co-clustering result evaluation unit is repeated, and when the predetermined condition is satisfied, the user based on the latest matrix information and the user The co-clustering apparatus according to claim 1, further comprising: a co-clustering end determination unit that stores a result of co-clustering related to an item in the storage unit. The user class update unit
From the latest matrix information, select one user to update,
Calculate the migration probability of each of the existing and new user classes to which the selected user is migrated,
Determine the user class to which the user is to be migrated according to the respective migration probability,
The information regarding the said user class is updated based on the said determination. The co-clustering apparatus of Claim 1 or Claim 2 characterized by the above-mentioned. The item class updating unit
From the latest matrix information, select one item to be updated,
Calculate the migration probability of each of the existing and new item classes to which the selected item is to be migrated,
Determine the item class to which the item is to be transferred according to the respective transfer probability,
The co-clustering apparatus according to claim 1 or 2, wherein information on the item class is updated based on the determination. In order to cluster item selection information indicating whether each user has selected each item or the frequency of selection, as matrix information related to the user and the item,
Based on a predetermined user class allocation probability formula, classify each of the users in the matrix information into a plurality of user classes,
Based on a predetermined item class allocation probability formula, classify each of the items in the matrix information into a plurality of item classes,
For each user / item block uniquely specified by the combination of the user class and the item class, a provisional selection probability that is a provisional probability that a user of the user class selects an item of the item class on the probability model Based on a predetermined item selection probability formula,
The matrix information is calculated by maximizing the posterior probability of the matrix information calculated from the predetermined user class allocation probability formula, the predetermined item class allocation probability formula, and the predetermined item selection probability formula. A co-clustering method using a co-clustering apparatus that performs clustering in both user units and item units,
A predetermined item class designation probability for each item class is given to each user class, a predetermined user class designation probability for each user class is given to each item class, and the predetermined item selection probability formula is A formula for calculating the provisional selection probability as a multiplication value or a value based on the multiplication value of the predetermined item class designation probability and the predetermined user class designation probability for each user / item block,
The co-clustering apparatus includes a storage unit that stores the item selection information and the matrix information.
The user class update unit receives the latest input of the matrix information, and determines the user class to which one or more of the users in the matrix information belongs based on the predetermined user class allocation probability formula and the provisional selection probability A user class update step to update;
The item class update unit receives the latest input of the matrix information, and determines the item class to which one or more of the items in the matrix information belong based on the predetermined item class allocation probability formula and the provisional selection probability An item class update step to be updated;
A co-clustering result evaluation unit calculates a posterior probability of the matrix information based on the latest matrix information, and a co-clustering result evaluation step;
A co-clustering method characterized by executing: The co-clustering end determination unit repeats the steps by the user class update unit, the item class update unit, and the co-clustering result evaluation unit until a predetermined condition is satisfied, and when the predetermined condition is satisfied, the latest matrix The co-clustering method according to claim 5, wherein a step of storing a result of co-clustering on the user and the item based on information in the storage unit is executed. The user class update unit, in the user class update step,
From the latest matrix information, select one user to update,
Calculate the migration probability of each of the existing and new user classes to which the selected user is migrated,
Determine the user class to which the user is to be migrated according to the respective migration probability,
The co-clustering method according to claim 5 or 6, wherein information on the user class is updated based on the determination. The item class update unit, in the item class update step,
From the latest matrix information, select one item to be updated,
Calculate the migration probability of each of the existing and new item classes to which the selected item is to be migrated,
Determine the item class to which the item is to be transferred according to the respective transfer probability,
The co-clustering method according to claim 5 or 6, wherein information on the item class is updated based on the determination.   A co-clustering program that causes a computer constituting the co-clustering apparatus according to any one of claims 1 to 4 to be executed.   A computer-readable recording medium on which the co-clustering program according to claim 9 is recorded.

Images