JP2018128708A - Tensor factor decomposition processing apparatus, tensor factor decomposition processing method and tensor factor decomposition processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To speed up the decomposition of nonnegative tensor factor and improve computation accuracy.SOLUTION: In a tensor factor decomposition processing apparatus 1 for performing factor decomposition processing of a tensor, a tensor construction unit 11 replaces a missing value of tensor constructed with multiple attribute information of a specific event as an element with a negative value. A matrix updating unit 21 updates elements of a factor matrix of the tensors processed in the tensor construction unit 11 by an updating expression based on distance between the tensor and the tensor calculated from the factor matrix. Moreover, if the element of the tensor is a negative value, it is determined that the element is a missing value of the tensor, and the correction value of the element is added to the update.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a data mining technique, and more particularly to a technique related to factorization for extracting a factor pattern from a plurality of attribute information, specifically to a technique for non-negative complex tensor factorization.

  As a technique for extracting a factor pattern from a plurality of attribute information, there is a technique called non-negative value tensor factorization and non-negative composite tensor factorization (Non-Patent Document 1). For these techniques, high-speed techniques have been proposed, and high-speed processing can be performed for sparse tensors (Patent Document 1).

  Furthermore, as an application of non-negative tensor factorization, there is a technique called non-negative tensor complementation that complements this missing value when part of attribute information is missing (Non-patent Document 2). Although non-negative tensor complementation is different in purpose, it can be said that non-negative tensor factorization adds processing for missing values.

Koh Takeuchi, Ryota Tomioka, Katsuhiko Ishiguro, Akisato Kimura, and Hiroshi Sawada, "Non-negative Multiple Tensor Factorization", ICDM, 2013, 1199-1204 Takashi Takeuchi, Tataka Naya, Nobuyoshi Ueda, “Nonnegative tensor interpolation using generalized KL divergence and its application to traffic flow analysis”, Proceedings of DEIM Forum, 2016 H7-2

Japanese Patent Laid-Open No. 2006-139391

  The conventional speed-up technique (Patent Document 1) is a speed-up technique that can be applied when the non-negative tensor is either “0-value sparse tensor” or “missing-value sparse tensor”. The “0 value sparse tensor” is a non-negative tensor with no missing value. The “missing value sparse tensor” represents a missing value of a non-negative tensor having a missing value as an element (zero element) having a value of “0”. With this technique, “0” as an observed value and a missing value cannot be expressed simultaneously in one tensor. Therefore, in order to process the non-negative (composite) tensor factorization used inside the non-negative (composite) tensor complementation at a high speed, there are the following problems.

  In order to apply the above conventional acceleration technology to tensors with missing values, the missing values must be replaced with zero elements in advance, and all zero elements must be treated as missing values. If there are zero elements that are not missing values, the calculation result will be inaccurate.

  On the other hand, if an accurate calculation is attempted without using the conventional speed-up technique, the calculation time increases as compared with the factorization of a tensor having the same scale and no missing value.

  In view of the above circumstances, an object of the present invention is to speed up non-negative tensor factorization and improve calculation accuracy.

  Therefore, one aspect of the present invention is a tensor factorization processing apparatus that performs tensor factorization processing, and replaces a missing value of a tensor constructed with a plurality of attribute information of a specific event as a negative value. Tensor construction means and matrix update means for updating the elements of the factor matrix of the tensor processed by the tensor construction means by an update formula based on the distance between the tensor and the tensor calculated from the factor matrix.

  One embodiment of the present invention is a tensor factorization processing method executed by a tensor factorization processing apparatus that performs tensor factorization processing, wherein a tensor deficiency constructed using a plurality of pieces of attribute information of specific events as elements. A tensor construction step that replaces the value with a negative value, and an update formula based on the distance between the tensor and the tensor calculated from the factor matrix of the factor matrix of the tensor processed in the tensor construction step There is a matrix update step to update.

  In one aspect of the matrix updating means and the matrix updating step, if the element of the tensor is a negative value, the element is determined to be a missing value of the tensor, and the correction value of the element is added to the update. .

  In one aspect of the tensor construction means and the tensor construction step, the constructed tensor is converted into a tensor having a lower order than the tensor at the time of the replacement.

  In addition, this invention can also be made into the aspect of the tensor factor decomposition processing program which makes a computer perform each step of the said method or the program which functions a computer as each means which comprises the said apparatus.

  According to the present invention described above, the non-negative tensor factorization can be speeded up and the calculation accuracy can be improved.

(A) is a block block diagram of the tensor factor decomposition processing apparatus in the embodiment of the present invention, (b) is a block block diagram of a tensor decomposition unit in the apparatus. (A) The block diagram which showed an example of the tensor based on log data, (b) is the list which illustrated the missing value information of the said tensor. The flowchart explaining the process of building a tensor. Explanatory drawing which illustrated the compression format of the tensor. The flowchart explaining the process which initializes the factor matrix of a tensor. The flowchart explaining the process which updates the factor matrix of a tensor.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to these embodiments.

[Overview]
The tensor factorization processing apparatus 1 illustrated in FIG. 1 introduces a representation of missing values that can be distinguished from zero elements by extending the speeding up technique of non-negative tensor factorization. In other words, using the fact that the elements of the original tensor are 0 or more and expressing the missing value of the tensor as a negative value element, the processing of the observed value and the missing value is performed in a single data scan. . For the denominator of the tensor update formula, first, an approximate value that is inaccurate but can be calculated at high speed is calculated, and then a correction value is calculated to ensure calculation accuracy. A high-speed calculation of the correction value is realized by performing a data scan for calculating the correction value together with the calculation of the update type numerator.

[Explanation of technical terms]
In describing the present embodiment, technical terms related to the present embodiment will be described.

  The attribute information represents a specific event by a combination of one or more attributes and a value corresponding to the combination. For example, attribute information indicating that people have visited a store can be represented by, for example, three attributes of a user ID, a store ID, and a day of the week, and the number of visits and the stay time corresponding to these attributes. The attribute information can be expressed as a tensor by regarding each attribute as a mode.

  A tensor is synonymous with a multidimensional array in this embodiment. For example, the third-floor tensor can be expressed as a three-dimensional array. However, the non-negative value tensor indicates a tensor in which all elements of the tensor are 0 or more.

  Mode refers to the tensor axis. For example, a matrix can be regarded as a tensor of the second floor, but at this time, there are two modes of a row direction and a column direction.

  The factor matrix is a matrix obtained by factoring the non-negative tensor, and there are as many as the modes.

  The missing value indicates a tensor element whose value is unknown. As a factor that a tensor includes a missing value, for example, in a case where data of a plurality of sensor states are collected for a certain period, a specific sensor has failed during a specific period, and the value of the data is unknown. . Missing values are distinguished from “0” as an observed value. “0” as an observation value corresponds to a case where “0” is output when, for example, a specific sensor is operating normally during a specific period and nothing is observed.

[Device configuration example]
The tensor factorization processing apparatus 1 includes an input data storage unit 10, a tensor construction unit 11, a tensor decomposition unit 12, a missing value estimation unit 13, and an output data storage unit 14.

  The input data storage unit 10 includes a non-negative tensor (hereinafter referred to as a tensor) on which factorization and missing value estimation are performed, missing value information indicating the position of the missing value of each tensor, and parameters used in the factorization. Saved. These pieces of information are assumed to be stored in advance in the input data storage unit 10.

  FIG. 2 shows an example of tensors and missing value information stored in the input data storage unit 10. The tensor of this example is the third floor tensor, and the three modes correspond to three pieces of attribute information “user ID”, “day of the week”, and “store ID”. A tensor element represents a value corresponding to a set of attribute information. For example, information that “user 2” visited “four times” at “store 3” on “Monday” can be expressed as a tensor element. A plurality of tensors may be provided by the technique of Non-Patent Document 1. The missing value information indicates which element of the corresponding tensor is missing.

  A tensor constructing unit (tensor constructing means) 11 constructs a tensor using a plurality of pieces of attribute information of a specific event as a mode in advance. The tensor is stored in the input data storage unit 10. Then, the tensor construction unit 11 extracts the tensor from the input data storage unit 10 and converts it into a format that can be efficiently scanned when factoring the tensor. Specifically, it is converted into a partial tensor that is a lower-order tensor than the extracted tensor, and non-zero elements (non-zero elements) among the elements are arranged in the main memory. When there is a missing value in an element in the tensor, the value of the element is replaced with an arbitrary negative value, and this is added as a non-zero element to the partial tensor of the tensor. Detailed processing will be described later.

  The tensor decomposition unit 12 performs factorization of the tensor obtained by the tensor construction unit 11. Detailed processing will be described later.

  The missing value estimation unit 13 estimates the missing value of the tensor based on the tensor factor matrix obtained by the tensor decomposition unit 12.

  The output data storage unit 14 stores the estimated value of the missing value obtained by the missing value estimation unit 13.

  In addition, the tensor decomposition unit 12 includes an initialization unit 20, a matrix update unit 21, and a calculation end evaluation unit 22, as illustrated in FIG.

  The initialization unit 20 performs an initialization process necessary for factorization of the tensor obtained by the tensor construction unit 11. Specifically, the elements of the tensor factor matrix are initialized with random numbers. Detailed processing will be described later.

  The matrix update unit (matrix update means) 21 updates the elements of the initialized factor matrix by an update formula based on the distance between the tensor obtained by the tensor construction unit 11 and the tensor calculated from the factor matrix. . Detailed processing will be described later.

  The calculation end evaluation unit 22 determines to continue the update based on the factor matrix updated by the matrix update unit 21. Specifically, the estimated value of the tensor is calculated from the factor matrix corresponding to each tensor, and the distance between the original tensor and the estimated tensor is calculated. However, since missing values cannot be compared, they are excluded from this calculation. A generalized KL divergence can be used for the tensor distance. The update is terminated when the distance satisfies the preset update end condition or when the update calculation count reaches a preset upper limit. On the other hand, when the distance does not satisfy the termination condition, the update is continued.

  The functional units 10 to 14 and 20 to 22 of the tensor factorization processing apparatus 1 described above are realized by hardware resources of a computer. That is, the tensor factorization processing apparatus 1 includes hardware resources related to a computer such as at least an arithmetic device (CPU), a storage device (memory, hard disk device, etc.), a communication interface, and the like. And these hardware resources cooperate with software resources (OS, application, etc.), and each function part 10-14, 20-22 is mounted. Moreover, you may make it each implement | achieve the function parts 10-14 and 20-22 in each computer.

[Process of tensor factorization of this embodiment]
The tensor factorization process of the present embodiment includes the following processes of “tensor construction (S100 to S104)”, “factor matrix initialization (S200 to S202)”, and “factor matrix update (S500 to S506)”. .

(Tensor construction)
The tensor construction step (S100 to S104) of this aspect will be described with reference to FIG. This process is executed by the tensor construction unit 11.

  S100: A tensor is extracted from the input data storage unit 10. In the following description, a case where there is one tensor will be described. However, a plurality of tensors may be provided depending on the problem setting of the tensor factorization processed by the apparatus. When there are a plurality of tensors, the processing after S100 is repeated by the number of tensors.

  S101: For each mode of the tensor, the processing after S102 is performed. When all modes have been processed, the process proceeds to end.

  S102: An array having the same dimension as that of the target mode is prepared. For example, a 10 × 8 × 6 third-order tensor has an array with 10 elements for the first mode, an array with 8 elements for the second mode, and 6 elements for the third mode. Prepare an array with.

  S103: The process of S104 is performed for each dimension of the target mode. When processing for all dimensions is completed, the process proceeds to S101. For example, when the first mode of the third floor tensor of 10 × 8 × 6 is set as the target mode, S104 is performed 10 times.

  S104: The partial tensor corresponding to the target dimension is expressed in a compressed format focusing on non-zero elements, and the reference is set to the corresponding element of the array. As this compression format, a format called Coordinate list (COO) can be applied. If the tensor is a third-order tensor, the partial tensor is a second-order tensor that is a lower-order tensor than the tensor, that is, a matrix. Therefore, a form called compressed row storage (CRS) is also applicable. However, in this aspect, when an element in the tensor has a missing value, the value of the element is replaced with an arbitrary negative value, and this is added to the compression format as a non-zero element.

  FIG. 4 exemplifies the flow of a series of processes in S104. (A) is a 3rd floor tensor to be processed. (B) represents (a) as a partial tensor, that is, a matrix column. (C) is a column of the compression format of each partial tensor. (D) is an example of the COO format and the CRS format when there is no missing value in the partial tensor. (E) is an example of COO format and CRS format when one element of the partial tensor of (d) is a missing value, and the element is regarded as a non-zero element having a negative value. Is shown.

(Factor matrix initialization)
The factor matrix initialization process (S200 to S202) will be described with reference to FIG. This process is executed by the initialization unit 20 of the tensor decomposition unit 12.

  S200: The rank number is extracted from the input data storage unit 10 as a parameter used in the factorization of this embodiment.

  S201: The process of S202 is executed for each factor matrix corresponding to the tensor obtained by the tensor construction unit 11 (S100 to S104). When the process is completed for all factor matrices, the process proceeds to the end.

  S202: Random numbers greater than 0 are substituted for all elements of each factor matrix that is the object. In the factor matrix, the row size is the size of the corresponding mode, and the column size is the rank number derived in S200.

(Update factor matrix)
The matrix update step (S500 to S506) of this aspect will be described with reference to FIG. This process is executed by the matrix update unit 21 of the tensor decomposition unit 12.

  S500: The process after S501 is performed on each element of the factor matrix initialized by the initialization unit 20. When the process is completed for all elements, the process proceeds to the end.

  S501: An approximate value is calculated for the denominator value of the update formula for updating the elements of the factor matrix. When the generalized KL divergence is used as the distance between tensors, the update formula is expressed as the following formulas (1) to (3).

  The tensor is assumed to be one third-order tensor for simplification, but the number may be any number of 1 or more and the number of ranks of 2 or more. Details of this updating formula are shown in Non-Patent Document 2.

  If this update equation is expanded assuming that there are no missing values in the tensor, that is, all elements are observed values, the above equation (4) is obtained. The details of this update formula are shown in Non-Patent Document 1, including the case where there are a plurality of tensors.

  The denominator value in the above equation (1) can be calculated at high speed by modifying the equation as in the above equation (5), assuming that there is no missing value in the tensor. However, when there is a missing value, the value of the denominator becomes inaccurate as the number of missing values increases. In this embodiment, this value is referred to as an approximate value.

  S502: The corresponding element of the tensor is scanned, and the processes after S503 are performed for each element. When the process is completed for all elements, the process proceeds to S506.

  The corresponding element is a non-zero element set when the index of the mode corresponding to the factor matrix to be updated is fixed and all the indexes of the remaining modes are arbitrary, as can be seen from the structure of the above formula (1). is there. For example, when the first mode is processed by a 10 × 8 × 6 third-order tensor, a maximum of 8 × 6 = 48 non-zero elements are included in this set.

  S503: If the element is a missing value, proceed to S504, otherwise proceed to S505. The determination that an element is a missing value can be performed by the sign of the value because the tensor construction unit 11 expresses the missing value as a non-zero element having a negative value.

  S504: The correction value of the denominator of the element is calculated and added to the correction value of the denominator of the update formula. The correction value of the element denominator is calculated by the above equation (6).

  S505: Calculate the numerator formula of the element and add it to the numerator of the update formula. The molecular formula of the element is the above formula (2).

  S506: The value of the update formula is calculated, and the elements of the factor matrix are updated. The denominator of the update formula is obtained by subtracting the correction value of S504 from the approximate value of S501. The renewal type numerator is the one obtained in S505. By substituting these into the above equation (1), the value of the update equation is obtained.

  The missing value estimation unit 13 estimates the missing value of the tensor based on the tensor factor matrix obtained by the update process by the tensor decomposition unit 12 described above. Then, the estimated value of the missing value is stored in the output data storage unit 14.

[Effect of this embodiment]
According to the tensor factorization processing apparatus 1 described above, in the tensor factorization used for non-negative (composite) tensor interpolation, the factorization corresponding to the missing value of the tensor can be speeded up. Therefore, while maintaining the accuracy of the calculation result, it is possible to suppress an increase in calculation time as compared with non-negative (composite) factorization for a non-negative tensor having no missing value with the introduction of a high speed technique.

  In particular, if the element of the tensor is a negative value, the calculation result of factorization of the tensor is determined by determining that the element is a missing value of the tensor and adding the correction value of the element to the update. Accuracy is ensured.

  As described above, even in situations where the factorization of tensors with missing values must be slow, by extending the conventional high-speed technology that does not support missing values, tensor factors without missing values As with decomposition, non-negative tensor interpolation can be speeded up.

[Other Embodiments of the Present Invention]
The present invention is realized by configuring a program that causes a computer to function as a part or all of each means (functional units 10 to 14, 20 to 22) constituting the tensor factorization processing apparatus 1 and causing the computer to execute the program. it can. Alternatively, a part or all of steps S100 to S104, S200 to S202, and S500 to S506 of the tensor factorization processing method executed by the apparatus 1 are configured by a program that is executed by a computer and is realized by causing the computer to execute the program. it can. These programs (tensor factor decomposition processing programs) can be provided by being stored in a known recording medium (for example, a hard disk, a flexible disk, a CD-ROM, etc.) that can be read by the computer. Alternatively, the program can be provided via the network via the Internet or e-mail.

  The above invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

DESCRIPTION OF SYMBOLS 1 ... Tensor factorization processing apparatus 11 ... Tensor construction part (tensor construction means)
DESCRIPTION OF SYMBOLS 12 ... Tensor decomposition part 13 ... Missing value estimation part 20 ... Initialization part 21 ... Matrix update part (matrix update means)
22 ... Calculation end evaluation section

Claims (7)

A tensor factorization processing apparatus that performs tensor factorization processing,
A tensor constructing means for replacing a missing value of a tensor constructed with a plurality of pieces of attribute information of a specific event as elements, and a negative value;
Matrix update means for updating an element of a factor matrix of a tensor processed by the tensor construction means by an update formula based on a distance between the tensor and a tensor calculated from the factor matrix. Tensor factorization processor.   The matrix updating means determines that if the element of the tensor is a negative value, the element is a missing value of the tensor, and adds the correction value of the element to the update. The tensor factorization processing apparatus according to 1.   The tensor factorization processing apparatus according to claim 1, wherein the tensor construction unit converts the constructed tensor into a tensor having a lower order than the tensor at the time of the replacement. A tensor factorization processing method executed by a tensor factorization processing apparatus that performs tensor factorization processing,
A tensor construction step of replacing a missing value of a tensor constructed with a plurality of pieces of attribute information of a specific event as elements, and a negative value;
A matrix update step of updating an element of the factor matrix of the tensor processed in the tensor construction step by an update formula based on a distance between the tensor and a tensor calculated from the factor matrix. Tensor factorization method.   5. In the matrix updating step, if the element of the tensor is a negative value, it is determined that the element is a missing value of the tensor, and the correction value of the element is added to the update. The tensor factorization processing method described in 1.   6. The tensor factorization processing method according to claim 4, wherein, in the tensor construction step, the constructed tensor is converted into a tensor having a lower order than the tensor at the time of replacement.   A tensor factor decomposition processing program for causing a computer to function as each means constituting the tensor factor decomposition processing device according to any one of claims 1 to 3.

Images