JP6278918B2 - Data analysis apparatus, method, and program - Google Patents

Description

  The present invention relates to a data analysis apparatus, method, and program.

  It is known that not only structured data such as purchase history represented by POS (Point of Sales) data but also many unstructured data such as text data and image data can be expressed in a matrix format by preprocessing. It has been. The usefulness of a technique called Non-Negative Matrix Factorization (NMF) has been shown so far as a technique for discovering clusters existing in these matrix-represented data (for example, non-patent literature) 1).

  By applying NMF, the input matrix data is decomposed into products of lower-order rank matrices. Each low-order matrix represents the contribution to the cluster of things corresponding to each row and each column, and cluster discovery is possible. Therefore, for example, a recommended product list for a user can be created based on a cluster extracted by applying to purchase history data, and articles can be automatically classified from application results to a news article document set.

An example of application of NMF to purchase data is shown in FIG. The user product matrix X = {x _ij } representing purchase data is a matrix of i rows and j columns in which the number of products purchased by the user corresponding to the i th row in the matrix corresponding to the j th column is the value of x _ij. It is.

  By applying NMF to this user product matrix,

A user feature matrix A = {a _ir } of I row and R column and a product feature matrix B = {b _jr } of J row and R column are obtained.

  However, the symbol

Are similar to each other, and the superscript symbol T represents the transpose of the matrix. Further, the value of a _{ir represents} the degree of contribution (affiliation degree) of user i to cluster r, and the value of b _jr represents the degree of contribution of product j to cluster r.

  When attention is paid to the column corresponding to the cluster 1 of the user feature matrix A in FIG. 15, the values of the first row, the second row, and the third row corresponding to the user 1, the user 2, and the user 3 are larger than 0. You can see that This indicates that user 1, user 2, and user 3 belong to cluster 1. Further, focusing on the row corresponding to the cluster 1 of the product feature matrix B, the value of the column corresponding to the product of the first column of beer 1, the second column of beer 2, and the third column of beer 3 is greater than zero. It turns out that it is a value. This can be said to represent a feature of cluster 1 that the three words beer 1, beer 2, and beer 3 are easily purchased by the same user. Therefore, the products called beer 1, beer 2, and beer 3 are collectively referred to as the product features of cluster 1. Similarly, users belonging to cluster 1 are referred to as user characteristics of cluster 1. The product features and user features of cluster 1 are collectively referred to as the features of cluster 1. Thus, cluster extraction as shown in FIG. 16 can be performed based on the user feature matrix A and the product feature matrix B obtained by applying NMF. Note that the rank number of the product feature matrix corresponding to the total number of clusters is determined in advance before analysis.

Hiroshi Sawada, “Basics of Non-Negative Matrix Factorization NMF and its Application to Data / Signal Analysis”, IEICE Journal, Vol. 95, No. 9, pp. 829-833, 2012. K. Takeuchi, K. Ishiguro, A. Kimura, and H. Sawada, Non-negative Multiple Matrix Factorization, Proceedings of 23rd International Joint Conference on Artificial Intelligence (IJCAI2013), pp. 1713-1720, 2013

  However, it is assumed that the purchase data used as an input by the technology of Non-Patent Document 1 described above deals with only a purchase history associated with a user ID, which indicates who purchased what product. It is not envisaged to perform an analysis that combines purchase histories that are not associated with user IDs.

  In recent data analysis, there are many situations where both data associated with a user ID and data not associated with the user ID exist. Two examples are given below.

  The first example is a situation where there is a limitation on the data use period in a format that can identify an individual. This is a restriction for avoiding the existence of data that can identify an individual for an unnecessarily long period from the viewpoint of protecting personal information. FIG. 17 shows an example of data. In the example given here, for data from 2013.4.1 to 2013.9.30, the user ID that identifies the individual is removed because the usage period has passed, and the data cannot be used and is not associated with the user ID. The data after 2013.10.1 is data associated with the user ID that can use all the columns including the user ID that identifies the individual. Therefore, for example, for the period from 2013.4.1 to 2013.9.30, only statistical information for each attribute such as the number of purchases of each product by male or female users as a whole can be used.

  The second example is an example of sharing data after anonymizing data between companies. FIG. 18 shows an example of data. The company, the data holder 1 and the data holder 2 store the data associated with the user IDs of their own data on the common data base. Although the company cannot extract the data itself of the data holder 1 and the data holder 2 from the common data base, the data that is calculated using the company-wide data and is not linked to the user ID, for example, the statistical information for each generation Can be used. Therefore, the data that can be used by the company is statistical information that is not associated with the user ID that the user has from the beginning and the user ID that can be extracted from the common data base.

  Even in the case where both data associated with the user ID and purchase data not associated with the user ID exist, the method of Non-Patent Document 1 cannot perform an analysis combining the purchase data not associated with the user ID. Therefore, analysis is performed without using this.

  However, the above approach has a problem when the ratio of data associated with the user ID becomes smaller than the total number of data. This is because, for example, when there is a situation where the product with the highest purchase quantity when using both data and the product with the highest purchase quantity when using only the data associated with the user ID, etc. It is because it becomes impossible to grasp.

  The present invention has been made in order to solve the above problems, and includes a data analysis apparatus, method, and program capable of analyzing the characteristics of the entire population, including the characteristics of the population that cannot be individually identified. The purpose is to provide.

In order to achieve the above object, the data analysis apparatus according to the first invention includes an individual i (1 ≦ i ≦ I, I is an integer of 1 or more) and an object included in a first individual group that can identify each individual. j (1 ≦ j ≦ J, where J is an integer _equal to or greater than 1), an I × J individual object matrix X having an element x _ij , and a second individual group that cannot be individually identified, An attribute object matrix Y of K × J having an element y _kj representing the degree of association between an object attribute k (1 ≦ k ≦ K, K is an integer of 1 or more) and the object j, and the individual i is a cluster The first feature matrix A of I × R having a non-negative element a _ir indicating that it belongs to r (1 ≦ r ≦ R, R is an integer of 1 or more), and the object j are included in the cluster r. a second feature matrix B J × R with elements b _jr nonnegative value indicating that it belongs the Sex k is a third feature matrix and C, and decomposed data analyzer of K × R with elements c _kr of non-negative values indicating that belonging to the cluster r, the first population An I × K individual attribute matrix V having an element v _ik representing the degree of association between the individual i and the attribute k included, and an element w representing the number of individuals having the attribute k for the second individual group based on the K-dimensional attribute vector W with _k, the ratio of the number of individuals with the attribute k for the number and the first population of individuals with the attribute k for the second population An attribute ratio information calculation unit for calculating an attribute ratio matrix P having an element p _kk representing the individual object matrix X, the attribute object matrix Y, the first feature matrix A, the second feature matrix B, and Based on the third feature matrix C, the The third feature matrix C is the first feature matrix A and the second feature under a linear constraint represented by the attribute ratio matrix P, the individual attribute matrix V, and the first feature matrix A. A matrix B and a feature matrix estimator that estimates the third feature matrix C, and an iterative determination unit that repeats the estimation by the feature matrix estimator until a predetermined iteration end condition is satisfied. Has been.

In the data analysis apparatus according to the first aspect of the present invention, element x _ij of the individual object matrix X is a non-negative value, element y _kj of the attribute object matrix Y is a non-negative value, and the first feature matrix A Element a _ir is non-negative, element b _jr of the second feature matrix B is non-negative, element c _kr of the third feature matrix C is non-negative, and the feature matrix estimator is The first feature matrix A, the second feature matrix B, and the third feature matrix C may be estimated by non-negative value decomposition.

The data analysis apparatus according to the second invention is a product j (1 ≦ j ≦ J, which is a user i (1 ≦ i ≦ I, I is an integer of 1 or more) included in a first user group that can be individually identified. I × J user product matrix X having an element x _ij indicating the number of purchases, where J is an integer equal to or greater than 1, and the user attribute k (1 ≦ k ≦) for the second user group that cannot be identified individually The user i has a cluster r (1 ≦ r ≦ R), and K × J attribute product matrix Y having an element Y _kj representing the number of purchases of the product j by users having K and K are integers of 1 or more. , R is an integer equal to or greater than 1), and an I × R user feature matrix A having a non-negative element a _ir and a non-negative element indicating that the product j belongs to the cluster r and product feature matrix B of J × R with b _jr, said attribute k is belonging to the cluster r A property characteristic matrix C of K × R with elements c _kr of non-negative values representing, in a decomposing data analyzer, the relevance between the attribute k and the user i included in the first user group Based on an I × K user attribute matrix V having an element v _ik and a K-dimensional attribute number vector W having an element w _k representing the number of users having the attribute k for the second user group. Attribute for calculating an attribute ratio matrix P having an element p _kk representing the ratio between the number of users having the attribute k for the second user group and the number of users having the attribute k for the first user group Based on the ratio information calculation unit, the user product matrix X, the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C, the attribute feature matrix C includes the attribute ratio Matrix P and previous A feature matrix estimator for estimating the user feature matrix A, the product feature matrix B, and the attribute feature matrix C under a linear constraint represented by the user attribute matrix V and the user feature matrix A; And an iterative determination unit that repeats estimation by the feature matrix estimation unit until the iterative completion condition is satisfied.

According to a third aspect of the present invention, there is provided a data analysis method in which an individual i (1 ≦ i ≦ I, I is an integer of 1 or more) and an object j (1 ≦ j ≦ J, An individual attribute k (1 ≦ k) for an I × J individual object matrix X having an element x _ij that represents the degree of association with J and an individual that is not distinguishable from each other. ≦ K, K is an integer greater than or equal to 1) and an attribute object matrix Y of K × J having an element Y _kj representing the degree of association between the object j, the individual i is represented by a cluster r (1 ≦ r ≦ R, R is an integer greater than or equal to 1) and a non-negative first element matrix A having a non-negative element a _ir indicating non-negative value a _ir and a non-negative value indicating that the object j belongs to the cluster r a second feature matrix B J × R with elements b _jr, the attribute k is the cluster r A third feature matrix C and the data analysis method in decomposing the data analysis device to the K × R with elements c _kr of non-negative values indicating that they belong, the attribute ratio information calculating unit, the first individual Represents an I × K individual attribute matrix V having an element v _ik representing the degree of association between the individual i and the attribute k included in the group, and the number of individuals having the attribute k for the second individual group Based on a K-dimensional attribute vector W having element w _k , the number of individuals having the attribute k for the second population and the number of individuals having the attribute k for the first population, A step of calculating an attribute ratio matrix P having an element p _kk representing a ratio of the same, and a feature matrix estimation unit, wherein the individual object matrix X, the attribute object matrix Y, the first feature matrix A, and the second feature Matrix B, and the third Based on the feature matrix C, the third feature matrix C is subject to linear constraints represented by the attribute ratio matrix P, the individual attribute matrix V, and the first feature matrix A. Estimating the matrix A, the second feature matrix B, and the third feature matrix C, and the estimation by the feature matrix estimation unit until the iteration determination unit satisfies a predetermined iteration termination condition. And repeating the steps.

  A program according to a fourth invention is a program for causing a computer to function as each part of the data analysis device described above.

According to the first and third inventions, the attribute ratio matrix P having the element p _kk representing the ratio is calculated based on the individual attribute matrix V and the attribute vector W, and the individual object matrix X, the attribute object Based on the matrix Y, the first feature matrix A, the second feature matrix B, and the third feature matrix C, the third feature matrix C includes an attribute ratio matrix P, an individual attribute matrix V, and a first feature. By including the first feature matrix A, the second feature matrix B, and the third feature matrix C under the linear constraint represented by the matrix A, the features of the indistinguishable individual group are included. Thus, the characteristics of the entire population can be analyzed.

According to the second invention, the attribute ratio matrix P having the element p _kk representing the ratio is calculated based on the user attribute matrix V and the attribute number vector W, and the user product matrix X, the attribute product matrix Y, the user Based on the feature matrix A, the product feature matrix B, and the attribute feature matrix C, the attribute feature matrix C is subjected to user features under linear constraints represented by an attribute ratio matrix P, a user attribute matrix V, and a user feature matrix A. By estimating the matrix A, the product feature matrix B, and the attribute feature matrix C, it is possible to analyze the features of the entire user group including the features of the user group that cannot be individually identified.

  According to the data analysis apparatus, method, and program of the present invention, it is possible to analyze the characteristics of the entire population including the characteristics of the population that cannot be individually identified.

It is a conceptual diagram which shows an example of the matrix obtained from the data which can be utilized from the data of own company, and the data of another company, and data. It is a conceptual diagram which shows an example of the analysis using the user goods matrix X. It is a conceptual diagram which shows an example of the matrix decomposition | disassembly which considered the relationship formed in the user goods matrix X and the attribute goods matrix Y. It is a conceptual diagram which shows an example of the relationship between the user product matrix X and the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C. It is a block diagram which shows the structure of the data analyzer which concerns on embodiment of this invention. 3 is a conceptual diagram illustrating an example of a user product information table stored in a storage unit 2. FIG. 3 is a conceptual diagram illustrating an example of an attribute product information table stored in a storage unit 2. FIG. 3 is a conceptual diagram illustrating an example of a user attribute information table stored in a storage unit 2. FIG. 3 is a conceptual diagram illustrating an example of an attribute number information table stored in a storage unit 2. FIG. 3 is a conceptual diagram illustrating an example of an attribute ratio information table stored in a storage unit 2. FIG. 4 is a conceptual diagram illustrating an example of a user feature table stored in a storage unit 2. FIG. 4 is a conceptual diagram illustrating an example of a product feature table stored in a storage unit 2. FIG. 3 is a conceptual diagram illustrating an example of an attribute feature table stored in a storage unit 2. FIG. It is a flowchart which shows the characteristic matrix estimation process routine in the data analysis apparatus which concerns on embodiment of this invention. It is a conceptual diagram which shows an example of a user product matrix, a user feature matrix, and a product feature matrix. It is a conceptual diagram which shows an example of cluster extraction. It is a conceptual diagram which shows an example in which the statistical information for every user cannot be used after a fixed period, and the statistical information for every attribute can be used. It is a conceptual diagram which shows an example of the data sharing between companies.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>

  First, an outline of the embodiment of the present invention will be described.

In the embodiment of the present invention, an I × J user product matrix X created from a purchase history associated with a user ID, a K × J attribute product matrix Y = created using a purchase history not associated with a user ID = {Y _kj }, K-dimensional attribute number vector W = {w _k } representing the number of users for each attribute existing in the attribute product matrix Y, I × K representing the correspondence between users existing in X and their attributes _Is used to extract a cluster from three matrices and one vector of user attribute matrix V = {v _ik }. Here, the user product matrix X is a product j (1 ≦ 1) by a user i (1 ≦ i ≦ I, I is an integer of 1 or more) included in each identifiable first user group given a user ID. j ≦ J, where J is an integer equal to or greater than 1) and is an I × J matrix having an element x _ij . In addition, the attribute product matrix Y indicates the number of purchases with the product j by the user having the user attribute k (1 ≦ k ≦ K, where K is an integer of 1 or more) for the second user group that cannot be individually identified. It is a K × J matrix with the elements y _kj represented. The user attribute matrix V is an I × K matrix having an element v _ik representing the degree of association between the user i and the attribute k included in the first user group. The attribute product matrix Y is a matrix that represents the total number of purchases of the product j having the element y _kj as the attribute k (for example, the entire male 30s). The element v _ik of the user attribute matrix V is a matrix that takes 1 when the user i belongs to the attribute k and 0 otherwise. The value of v _ik is not limited to 0 or 1, and may be 0 or a positive integer value. However, negative numbers are not used.

  An example of the creation of the above data is shown in FIG. As described above, the method of Non-Patent Document 1 cannot perform an analysis that combines purchase data that is not associated with a user ID, and therefore performs an analysis using only the user product matrix X as shown in FIG. Become.

On the other hand, in the embodiment of the present invention, using all the above-mentioned available data, a matrix decomposition method that considers the relationship that holds between the user product matrix X and the attribute product matrix Y as shown in FIG. is there. In FIG. 3, first, a ratio matrix of the number of users for each attribute existing in the user product matrix X and the number of users for each attribute in the attribute product matrix Y is defined as an attribute ratio matrix P = {p _kk } _{k = 1} ^K. To do. The values of the matrix P are calculated using the values of the user attribute matrix V and the attribute number vector W as shown in the figure, and values other than the diagonal component are zero values. Further, a statistical value for each attribute such as male or female calculated from the user product matrix X and the user attribute matrix V is called a partial statistical value, and a statistical value for each attribute indicated by the matrix Y is called an overall statistical value. Hereinafter, in the matrix decomposition model considered in the present embodiment to be described, modeling is performed on the assumption that the proportionality constant is an “approximate” proportional relationship in which the proportionality constant is the attribute ratio matrix P between the partial statistical value and the overall statistical value. The meaning of “approximately” mentioned here will be described later.

  In the embodiment of the present invention, the user product matrix X and the attribute product matrix Y

Consider the matrix decomposition form. C is referred to as an attribute feature matrix C = {c _kr } with K rows and R columns, and the value of c _kr represents the degree of contribution of attribute k to cluster r. In the embodiment of the present invention, a linear constraint that PV ^T A = C is established between the matrices A and C is introduced. A feature of the embodiment of the present invention is that a linear constraint is defined using the attribute ratio matrix P. By introducing this linear constraint, the feature matrices A, B, and C are obtained as an output considering the above-mentioned assumption that the partial statistics and the overall statistics are “approximately” proportional. Since the proportionality constant corresponds to the value of each element of the attribute ratio matrix P, there is an advantage that the proportionality constant itself can be estimated independently of the matrix decomposition of the user product matrix X and the attribute product matrix Y. FIG. 4 shows an application example of the embodiment of the present invention. The user feature matrix A represents the relationship between the user and the cluster, the product feature matrix B represents the relationship between the product and the cluster, and the attribute feature matrix C represents the relationship between the attribute and the cluster. Can be displayed. From this histogram, users, products, and attributes characteristic of each cluster can be identified.

  Where the sign

The meaning of the similar scale expressed in the above and the above-mentioned “approximately” proportional relationship will be explained. As described in Non-Patent Document 1 above, as a similar measure of a matrix, a measure based on Euclidean distance or a distance measure defined by generalized Kullback library divergence (KL distance) is used. The smaller the value, the more similar.

Which distance is used is determined in consideration of the properties of the data. For example, as described in Non-Patent Document 2, when the KL distance is used as the distance measure, each element x _ij of the user product matrix X is obtained according to the Poisson distribution of the parameter Σ _r a _ir b _jr. This is equivalent to considering a probability model. Therefore, the expected value of x _ij is Σ _r a _ir b _jr due to the Poisson distribution, but the modeling assumes that the value of x _ij in the actual data is deviated from the average. Has been. Based on this, when the term “approximately” proportional relationship is defined properly, in the embodiment of the present invention, the proportional relationship (proportional constant p _kk) between the expected value of partial statistics and the expected value of overall statistics for each attribute k. ) Is assumed. Therefore, by calculating the feature matrices A, B, and C after limiting the proportional relationship under the expected value, it is possible to perform an analysis in consideration of the value of the overall statistics.

  The general operation of the embodiment of the present invention is as follows.

  Step 1) Input user product matrix, attribute product matrix, user attribute matrix, and attribute number vector

  Step 2) Calculate the attribute ratio matrix

  Step 3) Estimate each feature matrix

  Step 4) Output each feature matrix

<Configuration of Data Analysis Device according to Embodiment of the Present Invention>

  Next, the configuration of the data analysis apparatus according to the embodiment of the present invention will be described. As shown in FIG. 5, a data analysis apparatus 100 according to an embodiment of the present invention includes a CPU, a RAM, and a ROM that stores a program and various data for executing a data analysis processing routine to be described later. Can be configured with a computer. Functionally, the data analysis apparatus 100 includes an input unit 1, a storage unit 2, a calculation unit 3, and an output unit 4 as shown in FIG.

The input unit 2 outputs a product j (1 ≦ j ≦) by a user i (1 ≦ i ≦ I, I is an integer of 1 or more) included in a first user group that can be individually identified and output from the external device 200. J, J is an integer equal to or greater than 1) I × J user product matrix X having an element x _ij representing the number of purchases, and a user attribute k (1 ≦ k) for a second user group that cannot be individually identified ≦ K, K is an integer greater than or equal to 1), and K × J attribute product matrix Y having an element Y _kj representing the number of purchases of the product j by a user, and the first user group that can identify each of them I × K user attribute matrix V having element v _ik representing the degree of association between user i and attribute k is received.

  In the storage unit 2, a user product information table 81 and an attribute product information table 82, which are created by each unit described later based on the user product matrix X, the attribute product matrix Y, and the user attribute matrix V received by the input unit 1. A user attribute information table 83, an attribute number information table 84, an attribute ratio information table 85, a user feature table 86, a product feature table 87, and an attribute feature table 88 are stored. Since the data in the table format can be expressed in a matrix format, in the following description, each table and each feature matrix are identified and used without distinction.

<User product information table 81>
As shown in FIG. 6, the user product information table 81 has a user ID field, a product ID field, and a purchase quantity field. In the user ID field, an identifier for identifying a user added by the user product information processing unit 30 is set. In the product ID field, an identifier for specifying a product purchased by the user added by the user product information processing unit 30 described later is set. In the purchase number field, 1 is set by the user product information processing unit 30 or the number of purchases of the user of the product is set. Although the value of the number of purchases can be set to 0 or a positive integer value, a negative number cannot be set.

<Attribute product information table 82>
As shown in FIG. 7, the attribute product information table 82 has an attribute ID field, a product ID field, and a purchase quantity field. In the attribute ID field, an identifier for specifying an attribute added by the attribute product information processing unit 32 described later is set. In the product ID field, an identifier for specifying a product added by the attribute product information processing unit 32 is set. In the purchase number field, 1 is set by the attribute product information processing unit 32, or the purchase number of the attribute of the product is set. In addition, although 0 or a positive integer value can be set as the value of the number of purchases, a negative number cannot be set.

<User attribute information table 83>
As shown in FIG. 8, the user attribute information table 83 has a user ID field, an attribute ID field, and an affiliation value field. In the user ID field, an identifier for identifying a user is set by a user attribute information processing unit 34 described later. In the attribute ID field, an identifier for specifying an attribute is set by the user attribute information processing unit 34. In the affiliation value field, 1 is set by the user attribute information processing unit 34 when the user belongs to the attribute, and 0 is set otherwise.

<Attribute Number Information Table 84>
As shown in FIG. 9, the attribute number information table 84 has an attribute ID field and a person value field. In the attribute ID field, an identifier for specifying an attribute is set by the attribute number information processing unit 36 described later. The number of users belonging to the attribute is set by the attribute number information processing unit 36 in the person number field.

<Attribute ratio information table 85>
As shown in FIG. 10, the attribute ratio information table 85 has an attribute ID field and a ratio value field. In the attribute ID field, an identifier for specifying an attribute is set by an attribute ratio information calculation unit 38 to be described later. A value calculated by the attribute ratio information calculation unit 38 is set in the ratio value field.

<User feature table 86>
As shown in FIG. 11, the user feature table 86 has a user ID field, a cluster ID field, and a user feature value field. An identifier for identifying the user is set in the user ID field by the feature matrix estimation unit 40 described later. In the cluster ID field, an identifier for identifying a cluster is set by the feature matrix estimation unit 40. In the user feature value field, the feature value of the user for the cluster calculated by the feature matrix estimation unit 40 is set.

<Product feature table 87>
As shown in FIG. 12, the product feature table 87 includes a product ID field, a cluster ID field, and a product feature value field. In the product ID field, an identifier for specifying a product by the feature matrix estimation unit 40 is set. In the cluster ID field, an identifier for identifying a cluster is set by the feature matrix estimation unit 40. In the product feature value field, the feature value for the cluster of the product calculated by the feature matrix estimation unit 40 is set.

<Attribute feature table 88>
As shown in FIG. 13, the attribute feature table 88 has an attribute ID field, a cluster ID field, and an attribute feature value field. In the attribute ID field, an identifier for specifying an attribute by the feature matrix estimation unit 40 is set. In the cluster ID field, an identifier for identifying a cluster is set by the feature matrix estimation unit 40. In the attribute feature value field, a feature value for the cluster of the attribute calculated by the feature matrix estimation unit 40 is set.

  The calculation unit 3 includes a user product information processing unit 30, an attribute product information processing unit 32, a user attribute information processing unit 34, an attribute number information processing unit 36, an attribute ratio information calculation unit 38, and a feature matrix estimation unit 40. And an iterative determination unit 42 and a feature matrix processing unit 44.

<User Product Information Processing Unit 30>
The user product information processing unit 30 stores the number of purchases for each user ID and each product ID in the user product information table 81 of the storage unit 2 based on the user product matrix X received by the input unit 1. In addition, when updating the user product information table 81, the user product information processing unit 30 adds a user ID field to the user product information table 81 stored in the storage unit 2 according to the added user, product, and number of purchases. , A line in which the values of the product ID field and the purchase quantity field are set is inserted. Note that the update timing by the user product information processing unit 30 may be manually managed by the system administrator based on data supplied from the external device 200, or when a new purchase occurs, for example. Alternatively, the external device 200 may automatically start processing.

<Attribute Product Information Processing Unit 32>
The attribute product information processing unit 32 stores the number of purchases for each attribute ID and each product ID in the attribute product information table 82 of the storage unit 2 based on the attribute product matrix Y received by the input unit 1. Further, when updating the attribute product information table 82, the attribute product information processing unit 32 adds an attribute ID to the attribute product information table 82 stored in the storage unit 2 according to the added attribute, product, and number of purchases. Insert a row in which the values of the field, the product ID field, and the purchase quantity field are set. Note that the attribute product information update timing by the attribute product information processing unit 32 may be manually managed by a system administrator based on POS data supplied from the external device 200, for example. When this occurs, the processing may be automatically started from the external device 200.

<User attribute information processing unit 34>
The user attribute information processing unit 34 stores the affiliation value for each user ID and each attribute ID in the user attribute information table 83 of the storage unit 2 based on the user attribute matrix V received by the input unit 1.

<Attribute Number Information Processing Unit 36>
The attribute number information processing unit 36 is based on a K-dimensional attribute number vector W having an element w _k representing the number of users having the attribute k for the second user group that cannot be individually identified. Is stored in the attribute number information table 84 of the storage unit 2. When the attribute number information table 84 is updated, the attribute number information processing unit 36 adds an attribute ID field to the attribute number information table 84 stored in the storage unit 2 according to the added attribute and the number of persons. And insert a line with the value of the person value field. Note that the user attribute information update timing by the attribute number information processing unit 36 may be manually managed by a system administrator based on POS data supplied from the external device 200, for example. The process may be automatically started from the external device 200 when it appears.

<Attribute ratio information calculation unit 38>
The attribute ratio information calculation unit 38, based on the user attribute matrix V and the attribute number vector W, counts the number of users having the attribute k for the second user group and the number of users having the attribute k for the first user group. An attribute ratio matrix P having an element p _kk representing the ratio is calculated.

Here, an I × K user attribute matrix V = {{v _ik }} _{i, k = 1} ^{I, K} is obtained from all data existing in the user attribute information table 83 stored in the storage unit 2. Further, the attribute number vector W = {w _k } _{k = 1} ^K is obtained from all data existing in the attribute number information table 84 stored in the storage unit 2.

Specifically, the attribute ratio information calculation unit 38 sets the attribute ratio matrix to P = {p _kk } _{k = 1} ^K , calculates the element p _{kk according} to the following equation (1), and calculates the ratio value for each attribute ID: It is stored in the attribute ratio information table 85 of the storage unit 2.

  For example, the attribute ratio information calculation unit 38 calculates each element of the attribute ratio matrix P according to the above equation (1), and stores it in the storage unit 2 according to the attribute of the attribute ratio matrix P and the value of the element. A row in which the value of the attribute ID field and the ratio value field is set is inserted into the attribute ratio information table 85. Note that the user attribute information update timing by the attribute ratio information calculation unit 38 may be manually managed by a system administrator based on POS data supplied from the external device 200, for example. The process may be automatically started from the external device 200 when it appears.

<Feature Matrix Estimation Unit 40, Iterative Determination Unit 42>
As will be described below, the feature matrix estimation unit 40 includes a user product matrix X, an attribute product matrix Y, a user attribute matrix V, an attribute ratio matrix P, a user feature matrix A stored in the user feature table 86, and a product feature table. Based on the product feature matrix B stored in 87 and the attribute feature matrix C stored in the attribute feature table 88, the attribute feature matrix C is expressed as an attribute ratio matrix P, a user attribute matrix V, and a user feature matrix A. The user feature matrix A, the product feature matrix B, and the attribute feature matrix C are estimated under the linear constraint of PV ^T A = C. Note that the processing of the feature matrix estimation unit 40 may be executed at an arbitrary timing, for example, when a feature output request is input from the external device 200 or by a predetermined periodic process.

Here, an I × J user product matrix X = {x _ij } _{i, j = 1} ^{I, J} is obtained from all data existing in the user product information table 81 stored in the storage unit 2. Further, a K × J attribute product matrix Y = {y _kj } _{k, j = 1} ^{K, J} is obtained from all data existing in the attribute product information table 82 stored in the storage unit 2. Further, an I × K user attribute matrix V = {v _ik } _{i, k = 1} ^{I, K} is obtained from all data existing in the user attribute information table 83 stored in the storage unit 2. Further, a K × K attribute ratio matrix P = {p _kk } _{k = 1} ^K is obtained from all data existing in the attribute ratio information table 85 stored in the storage unit 2. The user feature matrix A, the product feature matrix B, and the attribute feature matrix C are respectively A = {a _ir } _{i = 1, r = 1} ^{I, R} , B = {b _jr } _{j = 1, r = 1} ^{J , R 1} , C = {c _kr } _{k = 1, r = 1} ^{K, R.} The user feature matrix A represents the relationship between users and clusters, the product feature matrix B represents the relationship between products and clusters, and the attribute feature matrix C represents the relationship between attributes and clusters. I represents the total number of users, J represents the total number of products, and K represents the total number of attributes. Further, i corresponds to an identifier for specifying a user, j is an identifier for specifying a product, k is an identifier for specifying an attribute, and r is an identifier for specifying a cluster.

  Specifically, the feature matrix estimation unit 40 performs the following first to third processes.

First, the feature matrix estimation unit 40 performs, as a first process, a user product matrix X, an attribute product matrix Y, a user attribute matrix V, an attribute ratio matrix P, a user feature matrix A, a product feature matrix B, and an attribute feature matrix C. Based on the above, the element a _ir of the user feature matrix A is updated according to the following equation (2).

However,

And Here _{^ x ij} is the estimated value of _{x ij} by a user feature matrix A, and product feature matrix B, _{^ y kj} can be regarded as an estimate of the _{y kj} by property characteristic matrix C, and the product feature matrix B.

Then, the feature matrix estimation unit 40 stores the value of the element a _ir of the user feature matrix A updated by the above equation (2) in the user feature table 86.

Next, as a second process, the feature matrix estimation unit 40 performs the following expression (3) based on the user product matrix X, the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C. , Each element b _jr of the product feature matrix B is updated.

Then, the feature matrix estimation unit 40 stores the value of the element b _jr of the product feature matrix B updated by the above equation (3) in the product feature table 87.

Next, as a third process, the feature matrix estimation unit 40 performs each element of the attribute feature matrix C according to the following equation (4) based on the user attribute matrix V, the attribute ratio matrix P, and the user feature matrix A. c Update _kr .

Then, the feature matrix estimation unit 40 stores the value of the element c _kr of the attribute feature matrix C updated by the above equation (4) in the attribute feature table 88.

  The iteration determination unit 42 repeats the estimation by the feature matrix estimation unit 40 until the iteration end condition is satisfied. Specifically, if the maximum absolute value of the difference between the value of each element before the update and the value of each element after the update is larger than the variable δ indicating the maximum change width of the value update, the maximum value is set to δ. Assign a value. Note that the variable δ indicating the maximum change width of the value update is initialized for each series of update processing of each matrix. Further, the threshold value ε of the repetition condition and the maximum number of repetitions N are set in advance.

More specifically, the iterative determination unit 42 is updated by the feature matrix estimation unit 40 according to the element value a _ir ^old of the user feature matrix A before update stored in the user feature table 86 and the above equation (2). If the maximum value max _ir | _air ^old −a _ir ^new | of the difference between the element values a _ir ^new of the updated user feature matrix A is greater than δ, δ ← | _air ^old −a _ir ^new Update with |. The symbol “←” means a process of assigning the calculation result on the right side to the variable on the left side. Further, the value of the element of the user feature matrix A before the substitution process is described as a _ir ^old , and the value after the substitution process is described as a _ir ^new .

The iterative determination unit 42 then determines the maximum value max of the absolute value of the difference between the element value of the product feature matrix B before the update stored in the product feature table 87 and the value of the element of the product feature matrix B after the update. _jr | if is greater than _{δ, δ ← max jr | |} b jr old -b jr new b jr old -b jr new | to update. In addition, the value of the element of the product feature matrix B before the substitution process is described as b _jr ^old , and the value after the substitution process is described as b _jr ^new .

The iterative determination unit 42 then determines the maximum value max _kr of the absolute value of the difference between the element value of the attribute feature matrix C before update stored in the attribute feature table 88 and the value of the element of the attribute feature matrix C after update. If | c _kr ^old −c _kr ^new | is larger than δ, update as δ ← max _kr | c _kr ^old −c _kr ^new |. The value of the element of the attribute feature matrix C before the substitution process is described as c _kr ^old , and the value after the substitution process is described as c _kr ^new .

  The iterative determination unit 42 determines the feature matrix estimation unit 40 if the number of iterations n exceeds a predetermined maximum number of repetitions N or if δ representing the maximum change width due to update by the feature matrix estimation unit 40 is smaller than a predetermined threshold ε. The update process by is terminated.

Each of the updating formulas (2) to (4) is such that the left side and the right side are _equal when ^ x _ij = x _ij and ^ y _kj = y _kj for all users i, products j, and attributes k. In addition, it can be seen that the update stops because the value of the variable δ indicating the maximum change width of the update is equal to or less than the threshold ε. In addition, for a certain user i, when the above equation (2) is updated when ^ x _ij <x _ij and ^ y _kj <y _kj for all j and k, the numerator on the right side becomes larger than the denominator on the right side. Therefore, a _ir is updated to be larger than the current value, and the characteristic element a _ir is updated so that the values of ^ x _ij and ^ y _kj are increased.

  FIG. 14 shows a flowchart of the feature matrix estimation process in the feature matrix estimation unit 40 and the iteration determination unit 42.

  First, in step S100, the user feature matrix A stored in the user feature table 86, the product feature matrix B stored in the product feature table 87, and the attribute feature matrix C stored in the attribute feature table 88 are respectively obtained. initialize. Similarly, the threshold value ε of the end condition and the maximum number of repetitions are set.

  Next, in step S102, a variable δ indicating the maximum change width of the update used for determining the end condition is initialized.

In step S104, the user product matrix X stored in the user product information table 81, the attribute product matrix Y stored in the attribute product information table 82, the user attribute matrix V stored in the user attribute information table 83, the attribute The attribute ratio matrix P stored in the ratio information table 85, the user feature matrix A stored in the user feature table 86, the product feature matrix B stored in the product feature table 87, and the attribute feature table 88 are stored. Based on the attribute feature matrix C, the user feature matrix A is updated according to the above equation (2) and stored in the user feature table 86. Then, the element value a _ir ^old of the user feature matrix A before update stored in the user feature table 86 and the updated user feature matrix A updated by the feature matrix estimation unit 40 according to the above equation (2). If the maximum value max _ir | a _ir ^old −a _ir ^new | of the absolute value of the difference between the element values a _ir ^new is larger than δ, δ ← | a _ir ^old −a _ir ^new | is updated.

In step S106, based on the user product matrix X, the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C, the product feature matrix B is updated according to the above equation (3), and the product feature Store in table 87. Then, the maximum value max _jr | b _jr ^old − of the difference between the element value of the product feature matrix B before update stored in the product feature table 87 and the value of the element of the product feature matrix B after update. If b _jr ^new | is larger than δ, δ ← max _jr | b _jr ^old −b _jr ^new | is updated.

In step S108, the attribute feature matrix C is updated according to the above equation (4) based on the user attribute matrix V, the attribute ratio matrix P, and the user feature matrix A, and stored in the attribute feature table 88. Then, the maximum absolute value max _kr | c _kr ^old −c of the difference between the element value of the attribute feature matrix C before update stored in the attribute feature table 88 and the value of the element of the attribute feature matrix C after update. _{If kr} ^new | is larger than δ, δ ← max _kr | c _kr ^old −c _kr ^new | is updated.

  In step S110, the calculation repetition count is updated.

  In step S112, if the number of calculation iterations exceeds the maximum number of iterations determined in step S100, or if the variable δ representing the maximum change width due to the update is smaller than the threshold ε defined in step S100, the process ends.

<Feature matrix processing unit 44>
The feature matrix processing unit 44 outputs the value of each updated table stored in the storage unit 2 when the update process of the feature matrix estimation unit 40 is completed. For example, it may be executed when a feature output request is input from the external device 200. When all features are output, all the rows of the user feature table 86, the product feature table 87, and the attribute feature table 88 may be output. When only the product features of the cluster are used, for example, the product argument field and the product feature value field of the row having the cluster ID are output from the product feature table 87 using the request argument as the cluster ID, and then the product A product that characterizes the cluster can be obtained by displaying 10 product IDs in descending order of the value of the feature value field.

  The output unit 4 outputs each feature output by the feature matrix processing unit 44 to the external device 200.

As described above, according to the data analysis apparatus according to the embodiment of the present invention, the attribute ratio matrix P having the element p _kk representing the ratio is calculated based on the user attribute matrix V and the attribute number vector W. Based on the user product matrix X, the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C, the attribute feature matrix C is divided into an attribute ratio matrix P, a user attribute matrix V, and a user feature matrix. By estimating the user feature matrix A, the product feature matrix B, and the attribute feature matrix C under the linear constraint represented by A, including the features of the user group that cannot be individually identified, The effect that the feature can be analyzed is obtained.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, in the above-described embodiment, an example in which clusters are extracted from the user product matrix and the attribute product matrix is shown, but the present invention is not limited to this example.

For example, the first group of individuals that can be individually identified is other than the first user group, the second group of individuals that cannot be individually identified is other than the second user group, and the object is other than a product. And an individual i (1 ≦ i ≦ I, I is an integer of 1 or more) and an object j (1 ≦ j ≦ J, J is an integer of 1 or more) included in the first individual group that can identify each individual The individual object matrix X of I × J having the element x _ij representing the degree of association and the second individual group that cannot identify each individual attribute k (1 ≦ k ≦ K, where K is an integer greater than or equal to 1 an attribute object matrix Y of K × J with) an element y _kj representing the degree of relevance between the object j, the individual i is the cluster r (1 ≦ r ≦ R, R is belonging to an integer of 1 or more) a first feature matrix a I × R with elements a _ir of non-negative values indicating that the Obuji Transfected j is a second feature matrix B J × R with elements b _jr nonnegative value indicating that belonging to the cluster r, the attribute k is a non-negative value indicating that belonging to the cluster r It may be decomposed into a K × R third feature matrix C having an element c _kr .

  For example, an ID number is assigned to each of the visited place, the user, and the category, such as a matrix that represents the number of visits between the user and the visited place of the user and a set of matrices that represents the number of visits for each attribute in the visited place Can be identified and can represent data in a matrix format, and if there is a correspondence relationship between the user and the attribute to which the user belongs, everything is clustered by the data analyzer. Extraction is possible.

  Further, it is not necessary to be an integer like the number of appearances and the number of purchases, and generally a real number of 0 or more is sufficient. The method according to the embodiment of the present invention can also be applied when there are three or more input matrices.

  Further, the operation of each component of the data analysis apparatus shown in FIG. 5 of the above-described embodiment is constructed as a program and installed in a computer used as the data analysis apparatus to be executed or distributed via a network. It is possible.

DESCRIPTION OF SYMBOLS 1 Input part 2 Memory | storage part 3 Calculation part 4 Output part 30 User product information processing part 32 Attribute product information processing part 34 User attribute information processing part 36 Attribute number information processing part 38 Attribute ratio information calculation part 40 Feature matrix estimation part 42 Iterative determination Unit 44 Feature matrix processing unit 81 User product information table 82 Attribute product information table 83 User attribute information table 84 Attribute number information table 85 Attribute ratio information table 86 User feature table 87 Product feature table 88 Attribute feature table 100 Data analysis device 200 External device

Claims (5)

Degree of association between an individual i (1 ≦ i ≦ I, I is an integer of 1 or more) and an object j (1 ≦ j ≦ J, J is an integer of 1 or more) included in the first individual group that can identify each individual An individual object matrix X of I × J having an element x _ij that represents and an individual attribute k (1 ≦ k ≦ K, where K is an integer of 1 or more) for a second individual group that cannot be identified individually A K × J attribute object matrix Y having an element y _kj representing the degree of association with the object j,
A first feature matrix A of I × R having a non-negative element a _ir indicating that the individual i belongs to a cluster r (1 ≦ r ≦ R, R is an integer of 1 or more), and the object j Is a J × R second feature matrix B having a non-negative element b _jr indicating that it belongs to the cluster r, and a non-negative element c indicating that the attribute k belongs to the cluster r. a data analysis device that decomposes into a K × R third feature matrix C having _kr ,
An I × K individual attribute matrix V having an element v _ik representing the degree of association between the individual i and the attribute k included in the first individual group, and the attribute k for the second individual group. Based on a K-dimensional attribute vector W having an element w _k representing the number of individuals, the number of individuals having the attribute k for the second population and the attribute k for the first population are An attribute ratio information calculation unit for calculating an attribute ratio matrix P having an element p _kk representing a ratio with the number of individuals having the number of individuals,
Based on the individual object matrix X, the attribute object matrix Y, the first feature matrix A, the second feature matrix B, and the third feature matrix C, the third feature matrix C Under the linear constraints represented by the attribute ratio matrix P, the individual attribute matrix V, and the first feature matrix A, the first feature matrix A, the second feature matrix B, and the third feature. A feature matrix estimator for estimating the matrix C;
An iterative determination unit that repeats estimation by the feature matrix estimation unit until a predetermined iteration end condition is satisfied;
Data analysis device including The element x _ij of the individual object matrix X is non-negative, the element y _kj of the attribute object matrix Y is non-negative, the element a _ir of the first feature matrix A is non-negative, and the second The element b _jr of the feature matrix B of FIG. 6 is a non-negative value, the element c _kr of the third feature matrix C is a non-negative value,
The data analysis apparatus according to claim 1, wherein the feature matrix estimation unit estimates the first feature matrix A, the second feature matrix B, and the third feature matrix C by nonnegative decomposition. The number of purchases of a product j (1 ≦ j ≦ J, where J is an integer of 1 or more) by a user i (1 ≦ i ≦ I, I is an integer of 1 or more) included in the first user group that can be identified individually. I × J user product matrix X having the element x _ij represented, and a user attribute k (1 ≦ k ≦ K, where K is an integer equal to or greater than 1) for the second user group that cannot be identified individually A K × J attribute product matrix Y having an element Y _kj representing the number of purchases of the product j by the user,
An I × R user feature matrix A having a non-negative element a _ir indicating that the user i belongs to a cluster r (1 ≦ r ≦ R, R is an integer of 1 or more), and the product j is: A J × R product feature matrix B having a non-negative element b _jr representing belonging to the cluster r, and a K having a non-negative element c _kr representing that the attribute k belongs to the cluster r A data analysis device that decomposes into an attribute feature matrix C of × R,
An I × K user attribute matrix V having an element v _ik representing the degree of association between the user i and the attribute k included in the first user group, and the attribute k for the second user group. The number of users having the attribute k for the second user group and the attribute k for the first user group based on the K-dimensional attribute number vector W having an element w _k representing the number of users having An attribute ratio information calculation unit for calculating an attribute ratio matrix P having an element p _kk representing a ratio with the number of users having;
Based on the user product matrix X, the attribute product matrix Y, the user feature matrix A, the product feature matrix B, and the attribute feature matrix C, the attribute feature matrix C includes the attribute ratio matrix P and the user attribute. A feature matrix estimator for estimating the user feature matrix A, the product feature matrix B, and the attribute feature matrix C under linear constraints represented by a matrix V and the user feature matrix A;
An iterative determination unit that repeats estimation by the feature matrix estimation unit until a predetermined iteration end condition is satisfied;
Data analysis device including Degree of association between an individual i (1 ≦ i ≦ I, I is an integer of 1 or more) and an object j (1 ≦ j ≦ J, J is an integer of 1 or more) included in the first individual group that can identify each individual An individual object matrix X of I × J having an element x _ij that represents and an individual attribute k (1 ≦ k ≦ K, where K is an integer of 1 or more) for a second individual group that cannot be identified individually A K × J attribute object matrix Y having an element Y _kj representing the degree of association with the object j,
A first feature matrix A of I × R having a non-negative element a _ir indicating that the individual i belongs to a cluster r (1 ≦ r ≦ R, R is an integer of 1 or more), and the object j Is a J × R second feature matrix B having a non-negative element b _jr indicating that it belongs to the cluster r, and a non-negative element c indicating that the attribute k belongs to the cluster r. A data analysis method in a data analysis apparatus that decomposes into a K × R third feature matrix C having _kr ,
An attribute ratio information calculation unit includes an I × K individual attribute matrix V having an element v _ik representing a degree of association between the individual i and the attribute k included in the first individual group, and the second individual The number of individuals having the attribute k for the second individual group and the first individual based on a K-dimensional attribute vector W having an element w _k representing the number of individuals having the attribute k for the group Calculating an attribute ratio matrix P having an element p _kk that represents the ratio of the group to the number of individuals having the attribute k;
A feature matrix estimator is configured to generate the third object matrix X, the attribute object matrix Y, the first feature matrix A, the second feature matrix B, and the third feature matrix C based on the third feature matrix C. The feature matrix C is the first feature matrix A, the second feature matrix B, under linear constraints represented by the attribute ratio matrix P, the individual attribute matrix V, and the first feature matrix A. And estimating the third feature matrix C;
Repeating the estimation by the feature matrix estimation unit until the iteration determination unit satisfies a predetermined iteration termination condition;
Data analysis method including   The program for functioning a computer as each part of the data analysis apparatus of any one of Claims 1-3.