JP2018067227A - Data analyzing apparatus, data analyzing method, and data analyzing processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce load caused by estimation of posterior distribution.SOLUTION: A data analyzing apparatus according to the present embodiment has: parameter estimation means for regression-analyzing data to be analyzed to estimate a parameter which is a regression coefficient of an explanatory variable, Bayes estimation executing means for treating the parameter estimated by the parameter estimation means as prior distribution of Bayes estimation and for executing Bayes estimation on the basis of the prior distribution and a predetermined likelihood function to derive posterior distribution of the Bayes estimation; and sparse estimation executing means for executing sparse estimation by assigning, to loss function, a result of logarithmic transformation for the posterior distribution derived by the Bayes estimation executing means.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to a data analysis apparatus, a data analysis method, and a data analysis processing program.

  There is Bayesian estimation as a method for estimating parameters in an environment where data is available only sequentially. In the existing Bayesian estimation, since it is difficult to analytically obtain the posterior distribution, it is usually obtained by using an approximation such as MCMC (Markov chain Monte Carlo methods) (for example, see Non-Patent Document 1).

Yukito Iba et al., “Frontier of Statistical Computation 12 Computational Statistics II Markov Chain Monte Carlo Method and its Surroundings”, Iwanami Shoten, 2009. J. Friedman, T. Hastie and R. Tibshirani, “Regularization Paths for Generalized Linear Models via Coordinate Descent”, Journal of Statistical Software, Vol. 33, Issue 1, January 2010. Search August 25, 2016, Internet <https : //core.ac.uk/download/files/153/6287975.pdf>

  MCMC is a method for obtaining an optimal solution by repeatedly performing sampling. Therefore, MCMC takes time to calculate and cannot select a variable for minimizing input data (input variable). That is, in MCMC often used in Bayesian estimation, the dimension of variables cannot be reduced. Considering application to practical service, we want to reduce the input variables to reduce the data input load, but the conventional method is not suitable.

  An object of the present invention is to provide a data analysis device, a data analysis method, and a data analysis processing program that can reduce the load applied to the estimation of the posterior distribution.

  In order to achieve the above object, a first aspect of a data analysis apparatus according to an embodiment of the present invention includes: a parameter estimation unit that performs regression analysis of data to be analyzed and estimates a parameter that is a regression coefficient of an explanatory variable; Bayesian estimation execution means for deriving a posterior distribution of Bayesian estimation by performing a Bayesian estimation based on the parameter estimated by the parameter estimating means and performing Bayesian estimation based on the prior distribution and a predetermined likelihood function; There is provided an apparatus having sparse estimation execution means for executing sparse estimation by substituting a logarithmic transformation result for a posterior distribution derived by Bayesian estimation execution means into a loss function.

  According to a second aspect of the data analysis apparatus configured as described above, in the first aspect, the parameter estimation unit is such that the absolute value of the regression coefficient of the regression equation derived by the sparse estimation execution unit satisfies a predetermined condition. There is provided an apparatus for classifying the data to be analyzed using the explanatory variables, and performing regression analysis on the classified data to estimate a parameter which is a regression coefficient of the explanatory variable.

  A first aspect of a data analysis method according to an embodiment of the present invention is a method applied to a data analysis device, which performs regression analysis on data to be analyzed, estimates a parameter that is a regression coefficient of an explanatory variable, The estimated parameter is a prior distribution of Bayesian estimation, and a Bayesian estimation posterior distribution is derived by performing Bayesian estimation based on the prior distribution and a predetermined likelihood function, and a logarithmic transformation result for the derived posterior distribution A method for performing sparse estimation by substituting into a loss function is provided.

  According to a second aspect of the data analysis method described above, in the first aspect, the analysis target data is obtained using an explanatory variable whose magnitude is such that the absolute value of the regression coefficient of the derived regression equation satisfies a predetermined condition. There is provided a method of classifying and classifying the classified data and estimating a parameter which is a regression coefficient of an explanatory variable.

  An aspect of the data analysis processing program in the embodiment of the present invention is a program used in a computer that operates as a part of the data analysis apparatus in the first or second aspect, and the computer is used as the parameter estimation unit, the Bayes. An estimation execution unit and a program for functioning as the sparse estimation execution unit are provided.

  According to the present invention, it is possible to reduce the load for estimating the posterior distribution.

The figure which shows the structural example of the data analyzer in the 1st Embodiment of this invention. The flowchart which shows an example of the process sequence by the data analyzer in the 1st Embodiment of this invention.

Embodiments according to the present invention will be described below.
(First embodiment)
First, the first embodiment will be described. In the first embodiment, paying attention to the fact that the formula that takes the log (logarithm) of the posterior distribution of Bayesian estimation is the same format as the formula that takes the log of sparse estimation, the output that is the posterior distribution of Bayesian estimation is output. To perform sparse estimation.

  In the first embodiment, instead of MCMC, the regularized maximum likelihood method for estimating the mode value of the posterior distribution (that is, the value that maximizes the density function of the posterior distribution) is used as the Bayesian likelihood function. Sparse estimation. This sparse estimation is an estimation based on a Lasso (least absolute shrinkage and selection operator) type regularization method for realizing stable prediction using only effective explanatory variables for predicting an objective variable. Here, since a normal distribution is assumed as the prior distribution of Bayesian estimation, the sparse estimation algorithm can be used as it is for selecting effective parameters. The detailed algorithm is described below.

In the first embodiment, the parameter θ, which is the regression coefficient of each explanatory variable of the linear regression model, is accurately estimated in the prediction formula that can stably calculate the predicted value.
Let X (planning matrix) be the degree of connection before the event occurs, age sex, etc., and y (output (target estimated amount)) after the event has occurred. It is desirable to predict y from X in the linear regression model, but when the sample size is small, the estimated value of parameter θ, which is the regression coefficient of the explanatory variable of the linear regression model

Becomes unstable.

  Therefore, in the first embodiment, the estimated value of the parameter θ obtained from large-scale data such as a Web questionnaire.

For a more stable estimate

Calculate

  Estimated value even when the observation data itself used for estimation is small (for example, the sample size is small)

There is a Bayesian estimation as a method for calculating.

  In order to obtain a stable estimate, it is desirable to reduce the number of explanatory variables as much as possible in order to reduce the constraints on the variables required for the calculation. As a method for obtaining a stable estimate, sparse estimation There is.

  Therefore, in the first embodiment, a method for performing both Bayesian estimation and sparse estimation will be described. Specifically, the mode value of the posterior distribution is estimated by the following Bayesian estimation. An outline of the steps for performing both Bayesian estimation and sparse estimation is shown below.

(Step 1) Derivation of posterior distribution by assuming normality of prior distribution and data In Bayesian estimation, it is necessary to calculate the mode or average value of posterior distribution in order to derive the prediction formula to be finally obtained. is there. Here, calculation of the mode value of the posterior distribution will be described. This problem corresponds to the function maximization problem that is obtained by adding the logarithm of the posterior distribution to the log likelihood function. In the first embodiment, a sparse estimation penalty term that allows variable selection is added to this function. This penalty term is necessary for calculating a parameter corresponding to the regression coefficient of the explanatory variable of the prediction formula, and is necessary for obtaining the maximum value of the logarithm of the sparse likelihood penalty function described in Step 2 below.

(Step 2) Use sparse estimation algorithm to select valid parameters from mode of posterior distribution of Bayesian estimation. Use sparse estimation algorithm to calculate mode of posterior distribution of Bayesian estimation. Select only variables. Here, if the formula calculated in Step 1 above becomes a complicated formula, the optimization problem becomes difficult. In the first embodiment, the conventional algorithm used in Lasso is selected as an effective parameter. Because of this, it is in a form that can be used as it is, and it is possible to build a fast and simple program.

Details of Step 1 and Step 2 will be described below.
First, details of Step 1 will be described.
MCMC, which is an estimation method of the posterior distribution described above, is a general-purpose method that can be used for any distribution, but has a problem that the calculation time becomes long. Therefore, in the first embodiment, although not as versatile as MCMC, assuming normality in the prior distribution and data, the extremely fast Lasso algorithm can be used as it is for selecting effective parameters. .

  Specifically, the estimated value of parameter θ obtained from large-scale data such as a web questionnaire

Consider Bayesian estimation using. Here, the likelihood function, which is an index of goodness of fit to the data, is

And prior distribution of Bayesian estimation

(However, the logarithm of the likelihood function must be a quadratic function (normal distribution, etc.)).

Is

It becomes. The goal here is to find the parameter θ that maximizes the left side of equation (1). Therefore, it is necessary to calculate the left side of Equation (1), but it is difficult to directly obtain this. On the other hand, the right side of Equation (1) can be easily calculated because it is given by the product of the likelihood function and the prior distribution of Bayesian estimation. Therefore, consider calculating the right side of equation (1).

  Where the likelihood function

Follows a normal distribution, prior distribution

Is the mean vector

Suppose that it follows the normal distribution of. The reason for assuming a normal distribution in this way is that a regression model that assumes normality is the most standard method, and it is easy to calculate a Lasso estimated value that will be described later. In the likelihood function and prior distribution, as an average vector, the estimated value of parameter θ obtained from large-scale data such as a web questionnaire

Is used.

  Since the above equation (1) is an exponential function, it is difficult to calculate as it is. So, taking the logarithm of both sides of equation (1)

Is obtained. The result is a normal density function

The following formula (2) is obtained by substituting.

  However, τ (adjustment parameter) is a ratio of σ and γ as dispersion. C is a constant. Thus, assuming a normal distribution for the likelihood function and the prior distribution, the logarithm of the posterior distribution is given by a quadratic function with respect to the parameter β.

Next, details of Step 2 will be described.
Here, it is assumed that the parameter θ obtained from large-scale data such as a Web questionnaire is estimated by the L1 regularization method (Lasso). This is the estimated value of parameter θ performed in Step 1.

Lasso estimates using

Means to calculate

  First, attention is paid to the Lasso loss function in the above equation (2). The loss function is a function that represents the magnitude of the loss that the model does not apply to the data.

Is applicable. Since the Lasso loss function is given by a quadratic expression, if the loss function for the current Bayesian estimation is also given by a quadratic expression, the Lasso algorithm can be used as it is for selecting effective parameters. become. In this case, the negative log-likelihood function is

It becomes.

  In the first embodiment, paying attention to the characteristic that the loss function is a quadratic expression, the above negative log-likelihood function is used.

Subtract the logarithm of the prior distribution from this negative log-likelihood function

Is used. This corresponds to the above equation (2). The logarithmic subtraction of subtracting the logarithm of the prior distribution from the negative value means adding the prior distribution to Equation (2). By incorporating the parameter θ obtained from the Web data in the prior distribution in this way, Expression (2) is obtained.

  Next, the Lasso function is shown in the following equation (3).

  Substituting this equation (3) into the quadratic function shown in equation (2) yields the following equation (4).

  This equation (4) is the posterior distribution of Bayesian estimation.

Lasso with built-in. Here, the following equation (5) as a term other than the sum of absolute values of equation (4) is obviously a quadratic function related to the parameter β.

In this case, the Lasso algorithm can be used as is to select valid parameters.
The Lasso algorithm is described in Non-Patent Document 2, for example. Among these, the equation corresponding to the quadratic function related to the parameter β is the equation (4) of p.5 of Non-Patent Document 2, and the above equation (5) may be substituted into this equation.
This substitution will be described by applying to this embodiment. The logarithm of the posterior distribution of Bayesian estimation is proportional to the following equation (6).

The meaning of each variable in equation (6) is as follows.
f: Likelihood function π: Prior distribution θ: All parameters y: Output X (design matrix) → A (new explanatory variable) and y → Z (new objective variable). The following expression (7) is an expression for sparse estimation by Lasso (logarithm of likelihood function with penalties).

Thus, the least square estimator becomes the Bayes estimator of the original problem. p is a posterior distribution.
As described above, the equation (7) is equal to the equation (2) described above, and the following calculation is established within the parentheses of the equation (2).

  However,

It is.

The meaning of each variable is as follows.
β: Reciprocal of variance σ2 (corresponding to θ)
B0: Coefficient obtained from large-scale data (corresponding to θweb)
T: transposed matrix l: unit matrix According to the first embodiment, the Lasso algorithm shown in Expression (4) described in Step 2 can be used as it is for selecting effective parameters, and a prediction expression can be obtained. .
MCMC can also generate random numbers from the posterior distribution of Bayesian estimation by generating random numbers from the posterior distribution, but this is only the generation of random numbers from the distribution, and the posterior distribution of Bayesian estimation as shown in the outline of Step 1 It is not a method for obtaining the mode value of (ie, the value that maximizes the density function of the posterior distribution).
For this reason, even if MCMC is used, the mode value of the posterior distribution of Bayesian estimation cannot be accurately calculated. Of course, the mode value of the posterior distribution of Bayesian estimation can be approximated, but the variable cannot be selected because the regression coefficient value cannot be accurately estimated as zero as in Lasso. On the other hand, in the first embodiment, since the algorithm can use Lasso, the above-described problem that the variable cannot be selected can be solved.

  Further, in the first embodiment, the mode value of the Bayesian posterior distribution shown in the equation (1) can be obtained without performing repeated calculation by MCMC or the like, so that the prediction is made at a speed three orders of magnitude faster than MCMC, for example. An expression can be derived.

FIG. 1 is a diagram illustrating a configuration example of a data analysis apparatus according to the first embodiment of the present invention.
As shown in FIG. 1, the data analysis apparatus 10 according to the first embodiment of the present invention includes a database 1, a large-scale data parameter θ _Web estimation unit 2, a prior distribution input unit 3, a Bayesian estimation execution unit 4, a posterior distribution logarithm. A conversion unit 5 and a sparse estimation execution unit 6 are included.

  In the data analysis apparatus 10, the CPU built in the apparatus executes a program stored in advance in a storage device such as a hard disk to control the operation of each part of the electronic circuit, thereby causing each part to function. That is, in the data analysis apparatus 10, the CPU controls the operation of each part of the circuit in accordance with the instructions described in the control program, and various functions are realized by the software and hardware operating in cooperation. The database 1 is a device that stores data used for analysis, and is realized by a storage device such as a nonvolatile memory.

The large-scale data parameter θ _Web estimation unit 2 performs regression analysis on large-scale data stored in the database 1 and estimates a parameter θ _Web that is a regression coefficient of each explanatory variable.
Priors input unit 3 inputs the parameter theta _Web calculated by the large-scale data parameter theta _Web estimation unit 2 in the formula of the prior distribution of the Bayesian estimation.

The Bayesian estimation execution unit 4 performs Bayesian estimation using the prior distribution input by the prior distribution input unit 3.
The posterior distribution logarithmic conversion unit 5 obtains a logarithmically converted value of the posterior distribution output as a result of Bayesian estimation.
The sparse estimation execution unit 6 executes sparse estimation using the logarithmically transformed value as an input for sparse estimation by Lasso.

Θ _Web means the parameter for Web data,

Means the value estimated using θ _Web .

Next, a processing procedure by the data analysis apparatus 10 will be described. FIG. 2 is a flowchart illustrating an example of a processing procedure performed by the data analysis apparatus according to the first embodiment of the present invention.
First, the large-scale data parameter θ _Web estimation unit 2 reads large-scale data necessary for analysis stored in advance from the database 1 and uses it as a data set, and performs regression analysis on the target estimator y. The prior distribution input unit 3 sets the derived regression coefficient as a parameter as a result of the regression analysis.

As a prior distribution of Bayesian estimation (201 in FIG. 2).

  Next, the Bayesian estimation execution unit 4 uses the parameters input from the prior distribution input unit 3.

As a prior distribution, Bayesian estimation is executed based on a presumed likelihood function, and a posterior distribution of Bayesian estimation is derived according to Step 1 described above (202 in FIG. 2).
The posterior distribution logarithmic conversion unit 5 obtains a value obtained by logarithmically transforming the posterior distribution derived by the Bayesian estimation execution unit 4 as described in Step 2 (203 in FIG. 2).

  The sparse estimation execution unit 6 performs sparse estimation using the formula (4) obtained by substituting the loss function of the sparse estimation using Lasso into the logarithmic transformation result (formula (2)) (see formula (5)). (204 in FIG. 2).

  As a result, the data analysis apparatus 10 can derive the explanatory variables necessary for predicting the output y, its regression coefficient, and estimation accuracy (205).

  As described above, in the first embodiment, the posterior distribution is derived by providing normality to the prior distribution and the data, and a parameter effective for prediction is selected from the posterior distribution using the sparse estimation algorithm. The load for estimating the posterior distribution can be reduced.

(Second Embodiment)
Next, a second embodiment will be described.
In the second embodiment, in the first embodiment, a data set in which the observation data used in Bayesian estimation and the feature amount are similar is generated by paying attention to the absolute value of the regression coefficient. Then, by using the regression coefficient obtained by the regression analysis of this data set for the prior distribution of Bayesian estimation, the target regression equation is derived in a shorter time.

For example, in a group of men and women in their twenties, when they want to accurately predict people who like cakes, the characteristics obtained from psychology that women have more people who like cakes than men, The group can be predicted with higher accuracy.
However, if you want to accurately make an unknown prediction, such as predicting which political party you want to support for a group of men and women in their twenties, what attributes, for example, gender, We need a way to find out whether the job type is better or whether it is better to have a lover or a spouse.

In the second embodiment, a method for performing prediction using statistical knowledge will be described. Specifically, an explanatory variable (an example of the former) having a magnitude (for example, a predetermined value or more) with which the absolute value of the regression coefficient of the regression equation derived from the sparse estimation described in Step 2 of the second embodiment satisfies a predetermined condition The sample of the population is classified according to gender in
This sparse estimation has the property that the regression coefficient of the objective variable not related to the prediction of the objective variable to be obtained (corresponding to the person who likes cake in the former example) is zero or close to zero.

  Therefore, in the second embodiment, the data analysis apparatus 10 regards an explanatory variable having a regression coefficient that is not close to zero and has a large absolute value as an explanatory variable that is effective for predicting the objective variable, and the explanatory variable Classify groups. Then, the data analysis apparatus 10 performs the processing described in the first embodiment using the classified data sets.

Specifically, the large-scale data parameter θ _Web estimation unit 2 generates a data set having a feature quantity close to the data distribution introduced into the likelihood function from the large-scale data stored in the database 1. For data sets with similar features, regression analysis is performed on the data set used to calculate the output y, and large-scale data is obtained using data with the same value as the explanatory variable with a high regression coefficient derived as a result of this regression analysis. The large-scale data parameter θ _Web estimation unit 2 generates the classified data.

For example, by using a large-scale data of 3500 samples Web questionnaire (2,013.1-2 collection implementation), large-scale data parameter θ _Web estimation unit 2 estimates the large-scale data parameter θ _Web, collected in the actual workplace (2014.10～ A case will be described in which the 166 samples that have been collected 12) are used to predict what events will occur and interpersonal emotions (five levels) will change according to the Bayesian estimation performed by the Bayesian estimation execution unit 4.

  In this case, when the large-scale data is classified according to the type of event having a large absolute value of the regression coefficient, the prediction accuracy is improved by about 10% compared to the case where the classification is not performed in this way, and the Bayesian estimation is performed. It is possible to obtain a result that the number of samples required for deriving the prediction formula according to is about half to two thirds compared with the case of not classifying.

As described above, in the second embodiment, by classifying large-scale data by focusing on the regression coefficient, it is possible to improve prediction accuracy and derive an appropriate regression equation from a small amount of data.
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the methods described in the embodiments are, for example, magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD) as programs (software means) that can be executed by a computer (computer). , MO, etc.), a semiconductor memory (ROM, RAM, flash memory, etc.) or the like, or can be transmitted and distributed via a communication medium. Note that the program stored on the medium side includes a setting program that configures in the computer software means (including not only the execution program but also a table and data structure) to be executed by the computer. A computer that implements this apparatus reads a program recorded on a recording medium, constructs software means by a setting program as the case may be, and executes the processing described above by controlling the operation by this software means. The recording medium referred to in this specification is not limited to distribution, but includes a storage medium such as a magnetic disk or a semiconductor memory provided in a computer or a device connected via a network.

DESCRIPTION OF SYMBOLS 1 ... Database, 2 ... Large-scale data parameter (theta) _Web estimation part, 3 ... Prior distribution input part, 4 ... Bayes estimation execution part, 5 ... Post distribution logarithm conversion part, 6 ... Sparse estimation execution part.

Claims (5)

A parameter estimation means for performing regression analysis of the data to be analyzed and estimating a parameter that is a regression coefficient of the explanatory variable;
Bayesian estimation execution means for deriving a posterior distribution of Bayesian estimation by setting the parameter estimated by the parameter estimating means as a prior distribution of Bayesian estimation and performing Bayesian estimation based on the prior distribution and a predetermined likelihood function;
A sparse estimation execution means for executing a sparse estimation by substituting a logarithmic transformation result for the posterior distribution derived by the Bayesian estimation execution means into a loss function. The parameter estimation means includes
The analysis target data is classified using an explanatory variable whose magnitude of the regression coefficient of the regression equation derived by the sparse estimation execution means satisfies a predetermined condition, and the classified data is subjected to regression analysis. The data analysis apparatus according to claim 1, wherein a parameter that is a regression coefficient of an explanatory variable is estimated. A method applied to a data analysis device,
Analyze the data to be analyzed to estimate the parameters that are the regression coefficients of the explanatory variables,
The estimated parameter is set as a prior distribution of Bayesian estimation, and Bayesian estimation is performed based on the prior distribution and a predetermined likelihood function to derive a posterior distribution of Bayesian estimation,
A sparse estimation is performed by substituting a logarithmic transformation result for the derived posterior distribution into a loss function. The analysis target data is classified using an explanatory variable whose magnitude of the regression coefficient of the derived regression equation satisfies a predetermined condition, and the classification data is subjected to regression analysis, and the explanatory variable The data analysis method according to claim 3, wherein a parameter that is a regression coefficient is estimated. A program used for a computer operating as a part of the data analysis apparatus according to claim 1,
The computer,
The data analysis processing program for functioning as said parameter estimation means, said Bayesian estimation execution means, and said sparse estimation execution means.