JP6608721B2 - Data analysis apparatus and program - Google Patents

Description

The present invention relates to a data analyzer and program.

  In recent years, networking has progressed, and various data has been collected and accumulated via various devices. The various data are access information of the WEB site, customer purchase history, program recording / viewing history, customer age / sex information, and the like. Among them, recommendation services such as recommending products by clustering users according to attributes such as preferences using friendships of SNS (Social Networking Service), purchase history, and the like are performed. As a currently known clustering method, a method for obtaining a clustering result using spectral decomposition of a tensor has been proposed (see, for example, Non-Patent Document 1). In Non-Patent Document 1, an input matrix is expanded to a tensor, and a matrix that can be approximated by a product of three matrices is obtained by using singular value decomposition and CP decomposition that is a three-dimensional extension thereof. . By using those clustering results, a recommendation service or the like can be performed.

A Tensor Approach to Learning Mixed Membership Community Models, October 28, 2013

  In the prior art, since the input data is expanded to a tensor, the calculation memory and calculation time are in the order of the cube of the input data size, and the calculation cost is high. Also, symmetry is assumed for the input matrix, and full rank is assumed for the matrix representing the cluster structure. This is a severe limitation in applying to data that is actually acquired in business.

The present invention is to reduce the computational memory and provide infusion over data analyzer and programs provided constraints on input matrix.

The data analysis apparatus according to an aspect of the present invention includes an N-row and M-column basic matrix indicating the degree of association between each of the N first objects and the M second objects. A data analysis device for clustering the first object and the second object by decomposing the matrix into a second matrix and a third matrix, and for each element of the basic matrix, a basic matrix in which a value indicating the degree of association is input An acquisition unit for acquiring, a setting unit for setting K indicating the number of clusters of the first object and the second object, a dividing unit for dividing the basic matrix into at least three partial matrices, and each of the partial matrices On the other hand, using a result obtained by performing singular value decomposition using the largest singular value to K pieces, a transformation matrix generation unit for generating a transformation matrix, each of the partial matrices, and a transformation matrix corresponding to them An inner product meter that compresses the submatrix by calculating the inner product at And parts, using the compressed sub-matrix, a tensor generation unit generating a three rank tensor, tensor, decomposing decomposition unit using a CP degradation (CANDECOMP / PARAFAC decomposition), the decomposition result of the decomposition unit Using a calculation unit that calculates a first matrix, a second matrix, and a third matrix obtained by decomposing the basic matrix into three, and outputting at least one of the first matrix, the second matrix, and the third matrix, An output unit that outputs a clustering result of the first object and the second object.

  Note that these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. Also, any combination of recording media may be realized.

Data analyzer and program of the present invention, the amount of memory used for the calculation, and reduce the calculation time, it is possible to clustering without the constraint on the input data.

1 is a block diagram showing a schematic configuration of a data analysis system for executing a data analysis method according to Embodiment 1. FIG. 6 is an explanatory diagram illustrating an example of an input matrix according to Embodiment 1. FIG. 1 is a block diagram illustrating a schematic configuration of a data analysis apparatus according to a first embodiment. FIG. 3 is an explanatory diagram of a dividing unit of the data analysis apparatus according to the first embodiment. 6 is an explanatory diagram illustrating an example of an input matrix according to Embodiment 1. FIG. It is explanatory drawing which shows an example of the parameter obtained by performing the data analysis method based on the input matrix of FIG. 3 is a flowchart showing a flow of a data analysis method according to the first embodiment. 5 is a modification of the block diagram showing the configuration of the data decomposition apparatus according to Embodiment 1. FIG. FIG. 6 is a block diagram showing a schematic configuration of a data analysis apparatus according to a second embodiment. FIG. 10 is an explanatory diagram illustrating an input deformation unit of a data analysis device according to a second embodiment. FIG. 10 is an explanatory diagram illustrating a modification example of parameters in a parameter modification unit of the data analysis apparatus according to the second embodiment. 10 is a flowchart showing a flow of a data analysis method according to the second embodiment.

(Knowledge that became the basis of the present invention)
The inventor has found that the following problems occur with respect to the method described in the “Background Art” column.

  The conventional method needs to transform a matrix given as an input into a three-dimensional tensor format. For this reason, the amount of memory used and the calculation time at the time of calculation require the cube of the input data size. However, when dealing with large-scale data, there is a problem that the calculation cost is high and the calculation cannot be performed by placing the data in a memory. Also, it is assumed that the input matrix is a symmetric matrix and that the cluster structure potentially included in the input information is full rank. However, information that a certain user has purchased a certain product such as a purchase history becomes user × product in a matrix format, and is not symmetrical. Further, the fact that the cluster structure is full rank assumes that the number of clusters on the user side and the number of clusters on the product side are the same. This is difficult to think realistically, and is a very severe restriction in dealing with actual problems.

  In order to solve such a problem, a data analysis method according to an aspect of the present invention provides an N-by-M basic matrix indicating the degree of association between each of N first objects and M second objects. Is a data analysis method for clustering the first object and the second object by decomposing the first object, the second matrix, and the third matrix, and for each element of the basic matrix, An acquisition step of acquiring a basic matrix in which a value to be input is input; a setting step of setting K indicating the number of clusters of the first object and the second object; and dividing the basic matrix into at least three sub-matrices of the basic matrix A division matrix, a conversion matrix generation step for generating a conversion matrix using a result obtained by performing singular value decomposition using the largest singular value to K pieces for each of the partial matrices, Each and their corresponding Computation of inner product with a transformation matrix, inner product calculation step for compressing a submatrix, tensor generation step for creating a third-order tensor using the compressed submatrix, and tensor decomposition using CP decomposition A decomposition step, a calculation step for calculating a first matrix, a second matrix, and a third matrix obtained by decomposing the basic matrix into three using a decomposition result of the decomposition step; a first matrix, a second matrix, and a third matrix; An output step of outputting a clustering result of the first object and the second object by outputting at least one of the above.

  This enables calculation memory and calculation time in the square order of the data size without generating a large-scale tensor. Therefore, even large-scale data can be calculated on the memory.

  For example, when the basic matrix acquired in the acquiring step is not a symmetric matrix, an input deformation step of deforming into a symmetric matrix using a transposed matrix of the basic matrix may be included.

  As a result, even asymmetric input data can be calculated by being transformed into a symmetric shape.

  For example, using the result of the singular value decomposition calculated in the transformation matrix generation step, K may be set such that the sum of the singular values from the top is greater than or equal to a predetermined value with respect to the sum of all singular values.

  This makes it possible to automatically set K as the number of clusters.

  For example, using the result of the singular value decomposition calculated in the transformation matrix generation step, even if K is set such that the sum of squares from the top of the singular value is greater than or equal to a predetermined value with respect to the square sum of the entire singular value Good.

  This makes it possible to automatically set K as the number of clusters.

  For example, the decomposition step may be decomposed using singular value decomposition.

  Thereby, calculation using only singular value decomposition which is generally well known is possible.

  For example, when the clustering result of the first matrix and the third matrix calculated in the calculation step is out of the defined area, the first matrix and the third matrix are modified by changing the number of clusters so that the cluster is included in the defined area. A parameter transformation step may be included.

  This enables accurate clustering even when the cluster structure is not full rank.

  In addition, the data analysis apparatus according to the aspect of the present invention includes an N-row M-column basic matrix indicating the degree of association between each of the N first objects and the M second objects. , A data analysis device for clustering the first object and the second object by decomposing into a second matrix and a third matrix, wherein a value indicating a degree of association is input to each element of the basic matrix An acquisition unit that acquires a matrix, a setting unit that sets K indicating the number of clusters of the first object and the second object, a division unit that divides the basic matrix into at least three partial matrices of the basic matrix, and a partial matrix Using the result of singular value decomposition using from the largest singular value to the K singular values, a transformation matrix generation unit for generating a transformation matrix, each of the sub-matrices, and corresponding to them Performs inner product calculation with transformation matrix and compresses submatrix A calculation unit, a tensor generation unit that creates a third-order tensor using the compressed submatrix, a decomposition unit that decomposes the tensor using CP decomposition, and a basic matrix using the decomposition result of the decomposition unit And calculating at least one of the first matrix, the second matrix, and the third matrix, and calculating the first matrix, the second matrix, and the third matrix, And an output unit that outputs a clustering result of the second object.

  This enables calculation memory and calculation time in the square order of the data size without generating a large-scale tensor. Therefore, even large-scale data can be calculated on the memory.

  A program according to one embodiment of the present invention is a program for causing a computer to execute the above data analysis method.

  This enables calculation memory and calculation time in the square order of the data size without generating a large-scale tensor. Therefore, even large-scale data can be calculated on the memory.

  Note that these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. Also, any combination of recording media may be realized.

(Embodiment 1)
Hereinafter, embodiments will be specifically described with reference to the drawings. It should be noted that each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

[Overall system configuration]
FIG. 1 is a block diagram showing a schematic configuration of a data analysis system for executing the data analysis method according to the first embodiment.

  The data analysis system 1 receives an N-row M-column basic matrix indicating whether M objects (second objects) are related to each of N objects (first objects) as three inputs. A data analysis method for clustering users by decomposing into a matrix is executed. The target object represents a user or a product, and the basic matrix representing the degree of association is, for example, SNS friendship, “relevant” if the users are friends, and “unrelated” if they are not friends. , N objects and M objects represent the same thing, and an actual basic matrix is a symmetric matrix of N rows and N columns. Further, if the basic matrix is a purchase history, N objects are users, M objects are products, and a user purchases a product is “relevant”. Not relevant ".

  FIG. 2 is an explanatory diagram illustrating an example of a basic matrix.

  In the basic matrix shown in FIG. 2, N objects indicate whether or not each of the N objects is related. A value indicating whether or not there is a relationship is input to each element of the basic matrix. Specifically, “1” is assigned to elements that are related, and “0” is assigned to elements that are not related. For example, when the basic matrix shown in FIG. 2 is data representing the friendship relationship of SNS, “1” is input if the friendship relationship and “0” is input if it is not the friendship relationship. Note that the data values and formats are merely examples, and the present invention is not limited to these.

  Then, the data analysis system 1 clusters the objects by calculating the basic matrix decomposed into three matrices.

  Specifically, the data analysis system 1 includes an input device 200, a display device 300, and a data analysis device 400, as shown in FIG. The input device 200, the display device 300, and the data analysis device 400 are communicably connected via a network 500.

  The network 500 is a wired network such as Ethernet (registered trademark), a wireless network such as a wireless LAN, a public network, or a network in which these networks are combined. A public network is a communication line provided by a telecommunications carrier for communication of an unspecified number of users, and includes, for example, a general telephone line or ISDN.

  The input device 200 is a device to which a basic matrix of N rows and M columns is input. The input device 200 is, for example, a personal computer, a smartphone, a feature phone, a tablet terminal, or the like provided with an input unit 210 such as a keyboard, a touch panel, or a pointing device. When the basic matrix of N rows and M columns is input, the input device 200 transmits the basic matrix to the data analysis device 400 via the network 500.

  The display device 300 is a device that displays at least one matrix when at least one of the three matrices representing the clustering result is input from the data analysis device 400. The display device 300 is, for example, a personal computer, a smartphone, a feature phone, a tablet terminal, or the like provided with a display unit 310 such as a display. The analyst views at least one matrix displayed on the display unit 310 of the display device 300, so that the clustered result can be analyzed.

  In this embodiment, the case where the input device 200 and the display device 300 are independent and different terminals is illustrated, but the input device 200 and the display device 300 may be a single terminal. The input device 200, the display device 300, and the data analysis device 400 may be a single terminal, or may be connected without a network.

[Data analysis equipment]
The data analysis apparatus 400 is a processing apparatus that calculates three matrices as clustering results by using a basic matrix of N rows and M columns as an input. In the first embodiment, the input basic matrix is described as a symmetric matrix with N rows and N columns. The data analysis device 400 is, for example, a server, a personal computer, a smartphone, a feature phone, a tablet terminal, or the like.

  FIG. 3 is a block diagram illustrating a schematic configuration of the data analysis apparatus 400.

  As illustrated in FIG. 3, the data analysis device 400 includes an acquisition unit 410, a processing unit 420, and an output unit 430.

  The acquisition unit 410 acquires a basic matrix input from the input device 200 via the network 500 and outputs the basic matrix to the processing unit 420. The processing unit 420 is a processing unit that decomposes the basic matrix input from the acquisition unit 410 into three matrices, and includes a CPU, a RAM, a ROM, and the like. The processing unit 420 includes a setting unit 421, a dividing unit 422, a transformation matrix generation unit 423, an inner product calculation unit 424, a tensor generation unit 425, a decomposition unit 426, and a parameter calculation unit 427.

  The setting unit 421 stores the number of clusters used when performing clustering. For example, it is a set value such as dividing an object into K clusters. Note that the setting item may be a setting value input from the input device 200 even if it is not stored in the setting unit 421 in advance. In this case, the setting unit 421 stores the setting item received from the input device 200 via the acquisition unit 410. Further, as will be described later, it may be automatically determined by using the input basic matrix information.

The dividing unit 422 divides the input basic matrix into at least three partial matrices. As a specific example, a case where three partial matrices are generated will be described with reference to FIG. For the data size N of the basic matrix, four sets of X, A, B, and C で N are generated by an arbitrary method. Then, for the input basic matrix Gε {0,1} ^{N × N} , G _{X, A} ε {0,1} ^{X × A} , G _{X, B} ε {0,1} ^{X × B} , G Generate partial matrices G _{X, A} , G _{X, B} , G _{X, C} such that _{X, C} ∈ {0, 1} ^{X × C.}

  The transformation matrix generation unit 423 performs singular value decomposition on each of the partial matrices generated by the dividing unit, and generates a conversion matrix corresponding to each partial matrix using the decomposition results up to the upper Kth singular value. To do. Specifically, the conversion matrix generation unit 423 generates a conversion matrix corresponding to each partial matrix using Expressions (1) to (3).

  Here, K may be a value set by the setting unit 421, or, for example, up to the highest Kth singular value with respect to the sum of singular values or the sum of squares using the result of singular value decomposition. Alternatively, K may be set such that the sum or square sum exceeds a certain ratio (predetermined value). Here, the certain ratio is set as 70% or more, and may be freely determined as long as this range is satisfied.

The inner product calculation unit 424 compresses each partial matrix by inner product calculation of the partial matrix generated by the dividing unit and the transformation matrix generated by the transformation matrix generation unit. Specifically, X∈X, partial matrix G _X relative k ∈ _K, for the _A and transformation matrix W _A compresses and the inner product calculated by the equation (4).

However, V _A εR ^{X × K} and V _{A (x, k)} is an element in the xth row and the kth column.

とについては、式（５）、（６）で内積計算して圧縮する。 Similarly, submatrix G _{X, B} , G _{X, C} and transformation matrix

As for and, the inner product is calculated by the equations (5) and (6) and compressed.

  Thereby, the matrix which compressed each submatrix is produced | generated.

  The tensor generation unit 425 transforms the three compressed submatrices generated by the inner product calculation unit 424 into a third-order tensor format. Specifically, it is transformed into a tensor format using equation (7).

However, V _{A (x, k)} is a row vector of the x-th row of V _A.

  The decomposition unit 426 decomposes the tensor generated by the tensor generation unit 425 using CP decomposition. Since the CP decomposition is an extension of the singular value decomposition defined in the matrix so that it can be applied to a three-dimensional tensor, the same decomposition may be performed using the singular value decomposition. The decomposition result is calculated based on Expression (8) in a form including a part of three matrices that are parameters to be obtained.

The parameter calculation unit 427 calculates three matrices Π ^T , P, and あ る that are parameters to be obtained, using the result of decomposition by the decomposition unit. Specifically, the calculation is performed using Expressions (9) to (13).

  Note that if a stochastic block model (SBM) is assumed when obtaining Π, equation (14) may be used.

  If a mixed membership block model is assumed, equation (15) may be used.

  The output unit 430 outputs at least one of the first matrix, the second matrix, and the third matrix to the display device 300. The output unit 430 may output the basic matrix, the first matrix, the second matrix, and the third matrix in a lump, or may output them in combination. The output unit 430 may output the final product of the first matrix, the second matrix, and the third matrix as one piece of output information.

  FIG. 5 is an example of a basic matrix according to the present embodiment. FIG. 6 is an example of three matrices that are parameters according to the present embodiment.

For example, given a 9 rows and 9 columns of the fundamental matrix as shown in FIG. 10, when K = 3, as shown in FIG. 11, a first matrix of 9 rows and three columns corresponding to [pi ^T, 3 corresponding to P It is calculated as a second matrix with 3 rows and 3 columns and a third matrix with 3 rows and 9 columns corresponding to Π. For example, if the input matrix in FIG. 5 is SNS friendship data, the first matrix of parameters in FIG. 6 indicates that “user U1 belongs to cluster C1”. Thereby, it can be seen that the users belonging to the cluster C1 are U1, U4, and U6. Similarly, users belonging to the clusters C2 and C3 can also be known from the parameters. Regarding the third matrix, since the basic matrix is symmetric, the matrix has the same meaning as the first matrix. The second matrix means a matrix representing the relation strength between the cluster of the first matrix and the cluster of the third matrix. This second matrix means that the same cluster is strongly related, but has no relation to the different cluster.

[Data analysis method]
Next, a data analysis method according to the present embodiment will be described.

  FIG. 7 is a flowchart showing the flow of the data analysis method according to the present embodiment.

  In the input device 200, a value indicating whether or not there is a relationship is input to each element of the basic matrix. In the input device 200, a value K indicating the number of clusters is also input. After these inputs, the input device 200 outputs the basic matrix and K to the data analysis device 400. Note that if the setting item has already been set in the setting unit 421 of the data analysis device 400 and is used for the subsequent processing, it is not necessary to input the setting item with the input device 200.

  The acquisition unit 410 of the data analysis device 400 acquires the basic matrix and K input from the input device 200 via the network (step S101).

  The setting unit 421 stores K acquired by the acquisition unit 410 as a setting item (step S102).

  Next, the dividing unit 422 generates three partial matrices from the basic matrix (step S103).

  Next, the transformation matrix generation unit 423 performs singular value decomposition on the three sub-matrices generated in step S103, and uses the decomposition result using the top K singular values as a result, Three transformation matrices corresponding to the partial matrix are generated (step S104).

  Next, the inner product calculation unit 424 generates a matrix obtained by compressing the partial matrix by performing inner product calculation of the partial matrix generated in step S103 and the transformation matrix generated in step S104 (step S105).

  Next, the tensor generation unit 425 generates third-order tensor format data using the partial matrix compressed in step S105 (step S106).

  Next, the decomposition unit 426 decomposes the tensor generated in step S106 using CP decomposition (S107).

  Next, the parameter calculation unit 427 calculates parameters using the decomposition result of step S107 (S108).

  Finally, the output unit 430 outputs at least one of the parameters calculated in step S108 to the display device 300, and ends (S109).

  FIG. 8 is a block diagram showing a modification of the first embodiment. This data analysis method is possible even when there is no division unit in the first embodiment. For example, the set X, A, B, C⊂N used in the dividing unit 422 is replaced with N, N, N, N⊂N. In other words, the calculation is performed using the basic matrix as it is without dividing. Subsequent calculation processing can be similarly performed by replacing X, A, B, and C⊂N with N, N, N, and N⊂N. This is because when the basic matrix is divided, the calculation is performed without using the basic matrix, and there is a lack of information. This lack of information is effectively prevented by using all the basic matrixes. This is particularly effective when the data size of the basic matrix is small.

[Effects]
In the first embodiment, it is no longer necessary to create a large-scale tensor as in the prior art, and processing can be performed with a memory usage amount and a calculation time in the square order of the data size N. For this reason, even large-scale input data (basic matrix) can be calculated in the memory.

(Embodiment 2)
The data analysis method exemplified in the first embodiment is limited to a symmetric input basic matrix. Further, full rank property is assumed for the second matrix P that is a parameter. Therefore, it is impossible to handle a basic matrix such as a purchase history that is asymmetric and the second matrix is not full rank. With respect to such a basic matrix, a method of transforming the basic matrix into a symmetrical form and a method of calculating a second matrix that is not full rank will be described. In addition, about the same function as Embodiment 1, the same code | symbol is assigned and description is abbreviate | omitted.

  FIG. 9 is a block diagram illustrating a schematic configuration of the data analysis apparatus 401.

  The processing unit 440 includes an input deformation unit 428 and a parameter deformation unit 429 in addition to the processing unit 420 of the first embodiment.

  When the input basic matrix is asymmetric, the input deforming unit 428 uses the transposed matrix of the basic matrix to deform it into a symmetric shape. FIG. 10 is a diagram illustrating an example of a modification. As shown in Expression (16), the input basic matrix is transformed into a symmetrical form by using a transposed matrix of the basic matrix and a matrix filled with zeros.

  Subsequent calculations are performed using this symmetrically transformed basic matrix. Note that even if the input basic matrix is a symmetric matrix, it is possible to perform the calculation by performing this deformation, and therefore, the deformation may be performed even when it is symmetric.

  The parameter transformation unit 429 is configured such that when either the first matrix or the third matrix, which is the parameter calculated by the parameter calculation unit 427, is information outside the domain, the first matrix or Add or reduce new rows or columns to the third matrix and assign out-of-domain nodes to the new cluster. The definition area here is a condition that each object belongs to any one cluster. Belonging to two or more clusters or not belonging to any cluster is outside the domain of definition. Further, the second matrix is recalculated using the modified first matrix or third matrix. Specifically, the first matrix and the third matrix are transformed according to the equations (17) to (19).

  Here, the Unique function in Equation (18) is a function that deletes duplicate elements.

  FIG. 11 is an example showing a method of modifying the first matrix that is a parameter. It can be seen that the objects M7, M8, M9, and M10 of the first matrix before the deformation are out of the defined range. However, it can be seen that M7 and M8 are the same cluster, and M9 and M10 are the same cluster. Therefore, by adding two new clusters C4 and C5 and allocating them to each, a first matrix that can fit in the domain can be created. The same applies to the third matrix.

  FIG. 12 is a flowchart showing the flow of the data analysis method according to the second embodiment. In the input device 200, a value indicating whether or not there is a relationship is input to each element of the basic matrix. In the input device 200, a value K indicating the number of clusters is also input. After these inputs, the input device 200 outputs the basic matrix and K to the data analysis device 401. If the setting item has already been set in the setting unit 421 of the data analysis device 401 and is used for the subsequent processing, it is not necessary to input the setting item with the input device 200.

  The acquisition unit 410 of the data analysis device 401 acquires the basic matrix and K input from the input device 200 via the network (step S101).

  The setting unit 421 stores K acquired by the acquisition unit 410 as a setting item (step S102).

  Next, the input transformation unit 428 transforms the basic matrix into a symmetrical form using the transposed matrix of the basic matrix, and uses the deformed basic matrix as the basic matrix thereafter (step S110).

  Next, the dividing unit 422 generates three partial matrices from the basic matrix (step S103).

  Next, the transformation matrix generation unit 423 performs singular value decomposition on the three sub-matrices generated in step S103, and uses the decomposition result using the top K singular values as a result, Three transformation matrices corresponding to the partial matrix are generated (step S104).

  Next, the inner product calculation unit 424 generates a matrix obtained by compressing the partial matrix by performing an inner product calculation of the partial matrix generated in step S103 and the transformation matrix generated in step S104 (step S105).

  Next, the tensor generation unit 425 generates third-order tensor format data using the partial matrix compressed in step S105 (step S106).

  Next, the decomposition unit 426 decomposes the tensor generated in step S106 using CP decomposition (S107).

  Next, the parameter calculation unit 427 calculates parameters using the decomposition result of step S107 (S108).

  Next, when the first matrix or the third matrix is out of the domain, the parameter transformation unit 429 transforms it into a form that falls within the domain, and uses the transformed first matrix and third matrix, The two matrices are recalculated (S111).

  Finally, the output unit 430 outputs at least one of the parameters calculated in step S108 to the display device 300, and ends (S109).

[Effects]
According to the second embodiment, it is possible to accurately calculate a basic matrix in which the input basic matrix is asymmetric and the second matrix is not full rank. That is, clustering is possible using information of an asymmetric structure such as purchase history.

  In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Moreover, in each said embodiment, another process part may perform the process which a specific process part performs. Further, the order of the plurality of processes may be changed, and the plurality of processes may be executed in parallel.

  As described above, the data analysis method according to one or more aspects has been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  The present invention is useful as a data analysis method, data analysis apparatus, and program used for clustering. That is, the present invention can be applied in various fields that require clustering, such as a recommendation system and sentence classification.

DESCRIPTION OF SYMBOLS 1 Data analysis system 200 Input apparatus 300 Display apparatus 400, 401 Data analysis apparatus 410 Acquisition part 420,440 Processing part 421 Setting part 422 Division part 423 Transformation matrix production | generation part 424 Inner product calculation part 425 Tensor production part 426 Decomposition part 427 Parameter calculation part 428 Input transformation unit 429 Parameter transformation unit 500 Network

Claims (7)

The basic matrix of N rows and M columns indicating the relevance of each of the N first objects and the M second objects is decomposed into three first matrices, second matrices, and third matrices. A data analysis device for clustering one object and the second object,
An acquisition unit that acquires the basic matrix in which a value indicating the degree of association is input for each element of the basic matrix;
A setting unit for setting K indicating the number of clusters of the first object and the second object;
A dividing unit that divides the basic matrix into at least three partial matrices of the basic matrix;
For each of the sub-matrices, a transformation matrix generation unit that generates a transformation matrix using a result of performing singular value decomposition using the largest singular value to K pieces,
An inner product calculation unit that performs inner product calculation with each of the partial matrices and a conversion matrix corresponding to each of the partial matrices, and compresses the partial matrix;
A tensor generating unit that creates a third-order tensor using the compressed submatrix;
A decomposition unit that decomposes the tensor using CP decomposition (CANDECOMP / PARAFAC decomposition);
With decomposition result of the decomposing unit, a calculating unit in which the disassembled said first matrix into three fundamental matrix, calculates the second matrix and the third matrix,
An output unit that outputs a clustering result of the first object and the second object by outputting at least one of the first matrix, the second matrix, and the third matrix;
Data analysis device including. The data analysis apparatus according to claim 1, further comprising: an input transformation unit that transforms a symmetric matrix using a transposed matrix of the basic matrix when the basic matrix acquired by the acquisition unit is not a symmetric matrix. The conversion matrix generation unit sets K that causes a sum of singular values from a higher level to be equal to or greater than a predetermined value , based on the calculated result of singular value decomposition. The data analysis device described. The conversion matrix generation unit sets, for the sum of squares of the entire singular value, K that makes the sum of squares from the top of the singular value equal to or greater than a predetermined value, using the calculated singular value decomposition result. The data analysis apparatus according to 1 or 2. The data analysis device according to claim 1, wherein the decomposition unit decomposes using singular value decomposition. further,
When the clustering result of the first matrix and the third matrix calculated by the calculation unit is out of the definition area, the number of clusters is changed so as to enter the definition area, and the first matrix and the third matrix are transformed. The data analysis device according to any one of claims 1 to 5, further comprising a parameter transformation unit . The basic matrix of N rows and M columns indicating the relevance of each of the N first objects and the M second objects is decomposed into three first matrices, second matrices, and third matrices. A program for causing a computer to execute a data analysis method for clustering one object and the second object,
The data analysis method includes:
An acquisition step of acquiring the basic matrix in which a value indicating the degree of association is input for each element of the basic matrix;
A setting step for setting K indicating the number of clusters of the first object and the second object;
Splitting the base matrix into at least three sub-matrices of the base matrix;
A transformation matrix generation step for generating a transformation matrix using a result of performing singular value decomposition using the largest singular value to K pieces for each of the partial matrices;
An inner product calculation step of performing an inner product calculation with each of the partial matrices and a transformation matrix corresponding to each of the partial matrices, and compressing the partial matrix;
A tensor generation step of creating a third-order tensor using the compressed submatrix;
A decomposition step of decomposing the tensor using CP decomposition (CANDECOMP / PARAFAC decomposition);
A calculation step of calculating the first matrix, the second matrix, and the third matrix obtained by decomposing the basic matrix into three using the decomposition result of the decomposition step;
An output step of outputting a clustering result of the first object and the second object by outputting at least one of the first matrix, the second matrix and the third matrix;
Including programs.

Images