JP6484449B2 - Prediction device, prediction method, and prediction program - Google Patents

Description

  The present invention relates to a prediction apparatus, a prediction method, and a prediction program.

  Linear regression is a practical statistical method with a wide range of applications. Linear regression is used in various fields. For example, linear regression is used for power demand prediction and abnormality determination.

  In linear regression using a regression equation having a plurality of explanatory variables, there is a so-called multicollinearity problem in which the correlation between explanatory variables decreases the prediction performance. As one of the countermeasures, Ridge Regression is used. In ridge regression, regularization parameters need to be set appropriately to improve prediction accuracy.

  As the regularization parameter, it is desirable to set a parameter that reduces the estimated value of the prediction error. The estimated value of the prediction error can be calculated using, for example, AIC (Akaike's information criterion), BIC (Schwartz's Bayesian information criterion), cross validation, Mallow's Cp, and the like. In ridge regression, an appropriate parameter is obtained by searching for a regularization parameter that minimizes the estimated value of the prediction error.

  As a regularization parameter search method, several candidate values for regularization parameters are generated, the estimated values for each prediction error are calculated, and the candidate values for regularization parameters with the smallest prediction error value among them are calculated. The method of selecting is practical.

Arthur E. Hoerl and Robert W. Kennard, Ridge Regression: Biased Estimation for Nonorthogonal Problems, Technometrics. Vol.12, No.1 (Feb, 1970), pp. 55-67 Hui Zou and Trevor Hastie, Regularization and Variable Selection via the Elastic Net, Journal of the Royal Statistical Society: Series B (Statistical Methodology) .Volume 67, Issue 2, pages 301-320, April 2005

  However, the conventional technique has a problem that an appropriate regularization parameter cannot be efficiently searched.

  The regularization parameter has a possible range of 0 <λ ≦ ∞. Further, the regularization parameter has a non-linear effect on the prediction error. For this reason, in the search for the regularization parameter, trial and error is necessary in determining how to determine the candidate value for the regularization parameter, and an appropriate regularization parameter cannot be efficiently searched.

As a method for generating candidate values of regularization parameters whose effectiveness is empirically known, for example, a method of using exponentially increasing values such as 0.01, 0.1, 1, 10, and 100 is used. There is. In this method, for example, K _{i + 1} = k _i +1, and regularization parameter candidate values are 10 ^k1 , 10 ^k2 ,..., 10 ^k10 , and regularization parameter candidate values are generated. However, trial and error to determine the value of k ₁ is the initial value of the candidate value is required.

  Further, as described above, the upper limit of the regularization parameter candidate value is not limited. For this reason, when searching for a regularization parameter from a plurality of candidate values, the calculation time becomes longer as the number of candidate values increases. However, the number of candidate values is limited due to the limitation of calculation time. For this reason, it becomes a problem how to generate candidate values for the regularization parameter, and an appropriate regularization parameter cannot be efficiently searched.

  The present invention has been made in view of the above, and an object thereof is to provide a prediction device, a prediction method, and a prediction program capable of efficiently searching for an appropriate regularization parameter.

  In order to solve the above-described problems and achieve the object, the prediction device of the present invention has obtained the actual value of the objective variable of the regression equation of the ridge regression for obtaining the objective variable from the value of the explanatory variable, and the actual value. The value of the range that the regularization parameter of the regression equation can take is obtained by the storage unit storing the learning data in which the actual value of the explanatory variable at the time is stored and the change of the value of the predetermined range of the predetermined conversion parameter A generation unit that generates a plurality of conversion parameter candidate values within the predetermined range that can be taken by the conversion parameter of the conversion equation, and based on the learning data, the regularization parameter of the regression equation is converted by the conversion equation, and the generation A calculation unit that calculates an estimation value of the prediction error of the regression equation when each of the plurality of candidate values generated by the unit is applied to the conversion parameter; and an estimation of the prediction error calculated by the calculation unit But having a prediction unit that performs predicted by regression equation to which the candidate value having the minimum conversion parameters.

  The present invention has an effect that an appropriate regularization parameter can be efficiently searched.

FIG. 1 is a diagram schematically showing a matrix X and a column vector y. FIG. 2 is a diagram showing an example of the relationship between δ and λ. FIG. 3 is a diagram illustrating an example of a functional configuration of the prediction apparatus. FIG. 4 is a diagram illustrating an example of evaluation of a prediction result. FIG. 5 is a flowchart illustrating an example of the procedure of the prediction process. FIG. 6 is a diagram illustrating a computer that executes a prediction program.

  Embodiments of a prediction apparatus, a prediction method, and a prediction program according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[Linear regression and ridge regression]
First, linear regression and ridge regression will be described. In normal linear regression, for example, a regression coefficient is obtained by a least square method. Specifically, a regression coefficient that minimizes the sum of squares of the residual that is the difference between the regression equation and the observed value is obtained. This can be defined as a minimization problem as shown in the following equation (1).

min || Xβ-y || ² (1)

  Here, β is a regression coefficient. X represents an explanatory variable of the observed value in a matrix. y is a column vector of the objective variable of the observed value. Each row of the matrix X and the column vector y corresponds to one observation. Each column of the matrix X is an observation value type. FIG. 1 is a diagram schematically showing a matrix X and a column vector y. For example, in prediction of power demand, temperature, humidity, etc. are assigned to each column. If the number of observations is N and the type of observation value is P, the matrix X is a matrix of N rows and P columns. The column vector y is an N row column vector. “|| · ||” represents a square norm.

  Ridge regression is a method in which the magnitude of the regression coefficient is expressed by a square norm and the residual of the regression equation and the magnitude of the regression coefficient are squared simultaneously so that the regression coefficient does not increase. The regression coefficient of ridge regression can be obtained by solving a minimization problem as shown in the following equation (2).

^{min || Xβ-y || 2 +} λ || β || 2 (2)

  Here, λ is a regularization parameter and is a real number of 0 or more.

[Generalization of ridge regression]
Here, a generalized (extended) form of ridge regression will be described. In the Bayesian framework, ridge regression is positioned as a posterior probability maximization estimation when the prior distribution of regression coefficients is defined as a normal distribution. The above equation (2) corresponds to using an independent normal distribution for each regression coefficient as a prior distribution, but if a multivariate normal distribution is used as a prior distribution of regression coefficients, a generalized ridge regression is obtained. In this case, the regression coefficient can be obtained by solving the minimization problem shown in the following equation (3).

min || Xβ-y || ² + β ^T Λβ (3)

Here, Λ is a regularization parameter. When the matrix X is a matrix with N rows and P columns, the regularization parameter Λ is a symmetric matrix with P rows and P columns. “Β ^T ” represents the transpose of the regression coefficient β. In normal ridge regression, the regularization parameter is a single real value, whereas in generalized ridge regression, the regularization parameter is a matrix and consists of a plurality of real values.

  As another extension method, for example, Non-Patent Document 2 proposes Elastic Net. This is a method of expressing the magnitude of the regression coefficient in both the first norm norm. The regression coefficient can be obtained by solving the minimization problem shown in the following formula (4).

min || Xβ-y || ² + λ || β || ² + γ | β | (4)

  Here, γ is a regularization parameter unique to Elastic Net. “| • |” represents the first norm. The regularization parameter γ specific to Elastic Net is obtained by Least Angle Regression by specifying the number of regression coefficients that are non-zero. On the other hand, as for the regularization parameter λ, it is necessary to generate a plurality of candidate values and search for a parameter that minimizes the estimated value of the prediction error, as in the ridge regression.

[Standardization]
The term of the regularization parameter λ in the above equation (2) has a uniform effect on all the regression coefficients β. For this reason, the obtained regression coefficient β depends on the unit system (scale) of the explanatory variables. In order to obtain the same regression coefficient β for any unit system, standardization processing is necessary as pre-processing for ridge regression.

  Standardization is a process of converting each variable so that the average of each explanatory variable and objective variable is zero and the standard deviation is one. The standardization of the explanatory variable of the p-th column in the explanatory variable matrix X is calculated as shown in the following equation (5).

Here, subjected to _{x p} "¯" represents the average. The average x _p is the average of explanatory variables in the p-th column before standardization. The standard deviation σ _p is a standard deviation of explanatory variables in the p-th column before standardization. x _ip is an explanatory variable of the i-th row and the p-th column of the matrix X before standardization. x ′ _ip is an explanatory variable of the i-th row and the p-th column of the standardized matrix X. Standardize the objective variables in the same way. When standardization is performed, the constant term of the regression equation is always zero, so the constant term becomes unnecessary.

  When the value of the objective variable is calculated using the regression coefficient obtained after standardization, the value of the objective variable after standardization is returned to the original scale by the following equation (6).

Here, “￣” attached to y represents an average. The mean y is the mean of the objective variable before standardization. σ _y is the standard deviation of the objective variable before standardization. β ′ _p is a regression coefficient obtained after normalization.

[Conversion method]
The regression coefficient of the ridge regression is to solve the minimization problem of the above formula (2), and the regression coefficient β can be written as the following formula (7).

β = (X ^T X + λI) ⁻¹ X ^T y (7)

  Here, I is a unit matrix. When the matrix X is a matrix of N rows and P columns, I is a unit matrix of P rows and P columns.

  Thus, although the regression coefficient β of ridge regression is obtained analytically, the influence of the regularization parameter λ on the regression coefficient β is non-linear. For this reason, it is difficult to assume how the regression equation changes in the space of the objective variable when the regularization parameter λ is gradually increased from 0.

  Here, in order to intuitively understand the effect of the regularization parameter λ, the following formula modification is used. The above equation (7) can be rewritten as the following equation (8).

  If the explanatory variable and the objective variable are standardized, the equation (8) can be rewritten as the following equation (9).

Here, R _xx is a correlation matrix of explanatory variables. R _xy is a correlation matrix of explanatory variables and objective variables.

  For the regularization parameter, consider the following transformation (10).

  When the transformation of the equation (10) is applied to λ of the equation (9), the following equation (11) is obtained.

β = (δR _xx + (1-δ) I) ⁻¹ δR _xy (11)

Considering the case where there is no correlation between explanatory variables, R _xx is a unit matrix, and the regression coefficient is as shown in the following equation (12).

β = δR _xy (12)

  Further, the result of substituting the explanatory variables into the regression equation is as shown in the following equation (13).

y ^ (δ) = δXR _xy = δy ^ ^OLS (13)

Here, y ^ ^OLS is a regression result when the regression coefficient is obtained by a normal least square method, that is, when λ = 0. Note that OLS is an acronym for Ordinary Least Squares and refers to the ordinary least square method.

From this, it can be seen that when δ is moved at regular intervals between 0 and 1, y ^{^} (δ) moves at regular intervals between the normal regression result and y ^{^} (0) = 0. Therefore, if δ is searched at regular intervals from 0 to 1 as a regularization parameter, a linear search can be realized in a space in a range that the objective variable can take. Even if there is a correlation between explanatory variables, if the correlation is small, a property equivalent to this can be expected.

  Note that in the conversion by equation (10), when the range of δ is 0 <δ ≦ 1, δ = 1 and λ = 0, δ is increased when δ is decreased, and λ is infinite when δ approaches 0. It has the property of approaching large FIG. 2 is a diagram showing an example of the relationship between δ and λ. The example of FIG. 2 shows the relationship between δ in equation (10) and log (λ) when N = 1. As shown in FIG. 2, in the vicinity of δ = 0.5, it corresponds to exponentially changing λ, while the closer to δ = 0 or δ = 1, the log (λ ) Becomes larger. By this conversion, the influence of the change of δ on the prediction of λ is approximately linearly converted.

[Prediction method]
Next, a method for generating candidate values for the regularization parameter λ in normal ridge regression will be described. In the present embodiment, a regularization parameter conversion formula shown in the following formula (14), which is a generalization of the above formula (10), is used.

Here, k is a positive real number. The role of k is to adjust the size of the regularization parameter in accordance with the increase in the number of observations N since each element of X ^T X increases. The recommended value is k = N. When k = N, Expression (14) matches Expression (10).

In this embodiment, instead of generating a candidate value for the regularization parameter λ, a conversion parameter candidate value δ _i is generated as a regularization parameter by the following equation (15).

  Here, D is a positive integer determined in advance. i is an integer from 1 to D.

For each of the obtained D candidate values δ _i , an estimated value of the prediction error is calculated, and a candidate value δ _i that minimizes the estimated value of the prediction error is selected, thereby obtaining an appropriate regularization parameter. The recommended value of D is about 50 to 100.

  Next, a method for generating candidate values for regularization parameters in generalized ridge regression will be described. Since the regularization parameter of the generalized ridge regression is a matrix, it is necessary to set a value for each element of this matrix. The regression coefficient of generalized ridge regression can be written as the following equation (16).

β = (X ^T X + Λ) ⁻¹ X ^T y (16)

  Since the equation (16) is not significantly different from the normal ridge regression, the candidate value can be generated by using the following equation (17) instead of the equation (14).

Here, Λ _ij is an element of i row and j column of Λ. δ ^ij is a real number in the range 0 <δ ^ij ≦ 1. k has a role similar to that of Expression (14), and a recommended value is k = N.

Although the candidate values of the [delta] ^ij may also be generated by the formula (15), considering the combination of candidate values for each [delta] ^ij, the number of candidate values of the finally obtained Λ becomes exponential. If the candidate values for each δ ^ij are generated with random numbers in the range 0 <δ ^ij ≦ 1 instead of this method, the number of candidate values for Λ can be limited. A constraint may be imposed on the matrix of Λ. For example, only the diagonal component of Λ may be determined by Expression (17), and a constraint may be imposed such that other than the diagonal component is zero. Thereby, the number of candidate values of Λ can be limited.

[Advanced search method for δ]
Next, an advanced search method for δ will be described. The above-described method for generating δ candidate values at equal intervals is one of the regularization parameter search methods for minimizing the estimation value of the prediction error. In practice, the above method is often sufficient, but when ridge regression is applied to large-scale data with many types of observation values, regularization parameters may be set more strictly. For example, when applying ridge regression to large-scale data where the types of observations exceed 10,000, it may be necessary to set regularization parameters more strictly. In this case, a more advanced search method is useful for searching for a regularization parameter that minimizes the estimated value of the prediction error with as few candidate values as possible.

  As an advanced search method, a method of gradually narrowing down the range of the regularization parameter that minimizes the estimated value of the prediction error can be considered. As an example of an advanced search method, candidate values for δ are generated at equal intervals to calculate an estimation value of prediction error, and candidates for δ are calculated at a shorter interval for a predetermined range including a portion where the estimation value of the prediction error is minimum. There is a method of repeating a local search for generating a value and calculating an estimated value of a prediction error until a predetermined end condition is satisfied. This predetermined range should just be a range more than equal intervals, for example, shall be the range of equal intervals. As the end condition, for example, the local search may be repeated a predetermined number of times, or the estimated value of the prediction error may be equal to or less than a predetermined reference. For example, first, ten candidate values are generated at equal intervals in a range from 0 to 1, and an estimated value of the prediction error is calculated. Next, with respect to the range from 0 to 0.5 and the range from 0.5 to 1, the previous candidate value includes the one that minimizes the estimated value of the prediction error. Generate candidate values. By repeating this, it is possible to efficiently narrow down the range and efficiently search for a regularization parameter that minimizes the estimated value of the prediction error.

  In generalized ridge regression, it is useful to generate candidate values using random numbers, but as a more advanced method, a direct search method such as a local search or a genetic algorithm is also useful.

[Explanation variable selection method]
Next, a sequential explanatory variable selection method will be described. In ridge regression, the regression equation may have multiple explanatory variables. For example, the regression equation with P number of explanatory variables X _p can be expressed as the following equation (18).

  Here, y is an objective variable. β is a regression coefficient. In the equation (18), the objective variable y is calculated from regression equations of a plurality of explanatory variables.

For example, as the objective variable y power demand, when performing prediction of the power demand, the explanatory variable X _p are temperature, humidity, corresponding to the wind speed and the like. However, when creating a regression equation, it is not always necessary to use all the observed values, and there are cases where prediction can be made with sufficient accuracy using some explanatory variables. For example, even if temperature, humidity, and wind speed are observed, using only the temperature and wind speed in the regression equation may improve prediction accuracy or facilitate interpretation of the prediction result.

  A method for automatically selecting an explanatory variable to be incorporated into a regression equation when there are a plurality of explanatory variables in this way will be described. This technique starts with a regression equation with one explanatory variable and repeats adding explanatory variables one by one to the regression equation. For selecting the explanatory variable to be added, the one having the largest absolute value of the correlation between the explanatory variable and the error of the regression equation at that time is selected. The regression coefficient of the regression equation is calculated by ridge regression, and the regularization parameters are determined by the proposed method.

  An example of a specific procedure is shown below.

Step 1: Initialize an area for selecting an explanatory variable. For example, the area A ₀ = {} holding the selected explanatory variable is emptied. In addition, it initializes Y ^ ₀ = 0. Also, the counter t = 1 is initialized.

Step 2: For each unselected explanatory variable, a correlation with the error of the regression equation due to the selected explanatory variable is calculated. At first, since _A0 is empty and there is no selected explanatory variable, the predicted value of the objective variable becomes zero, and the actual value of the objective variable is directly calculated as an error. The error when the explanatory variable is selected is calculated in step 6 described later. For example, the correlation C _P between each explanatory variable and the error of the unselected is calculated from the following equation (19).

C _P = X (: | p) ^T (y−Y ^ _t−1 ) (19)

Here, X is a matrix of explanatory variables of N rows and P columns. A _t is the set of column number that points to the explanatory variables incorporated in the regression equation. X (: | p) is the p-th column vector of the matrix X. X (: | A) is a matrix having only X columns corresponding to the column numbers included in A. Y ^ ₀ is a column vector with N elements, and each element is 0.

Step 3: From the unselected explanatory variables, an explanatory variable having the maximum absolute value of the correlation calculated in Step 2 is obtained. That is, among the unselected explanatory variables, an explanatory variable having the most similar error and variation is obtained. For example, among the explanatory variables p that are not included in A _t−1 , the explanatory variable p having the maximum | C _P | is defined as p _t ^* .

Step 4: The absolute value of the correlation C _P incorporates a maximum of explanatory variables p _t ^* to regression. This can be expressed, for example, by the following equation (20).

_{A t = A t-1 ∪} {p t *} (20)

Equation (20) _indicates that the _{A t} by adding the explanatory variable _p ^{t *} to _{A t-1.}

Step 5: The regularization parameter of the regression equation is converted by the conversion equation to generate a candidate value δ _i of the conversion parameter δ, and an estimated value of the prediction error is obtained. Then, by applying the candidate value estimate of the prediction error is minimized in the conversion parameter, a regression coefficient beta _t, identifies the regression equation. The regression coefficient β _t represents the regression coefficient at the counter t.

Step 6: calculating an error of the predicted value and the actual value of the objective variable calculated from the regression equation according to the regression coefficient beta _t. The predicted value can be expressed as the following equation (21), for example.

_{Y ^ t = X (: |} A) β t (21)

  Step 7: It is determined whether the error satisfies a predetermined stop condition. If the stop condition is not satisfied, the counter t is incremented (t = t + 1), and the process returns to step 2. Thereby, steps 2 to 7 are repeated. When the stop condition is satisfied, the selection of the explanatory variable is finished. Thereby, the regression equation is specified. The stop condition is, for example, until a predetermined number of explanatory variables having a high correlation is selected, or until an error between the predicted value and the actual value of the objective variable calculated from the regression equation using the selected explanatory variable increases. When the stop condition is until a predetermined number of explanatory variables with high correlation is selected, prediction is performed using a regression equation when the stop condition is satisfied. If the stop condition is set to increase the error between the predicted value of the objective variable calculated from the regression equation with the selected explanatory variable and the actual value, the error will increase in the regression equation when the stop condition is met. Therefore, the prediction is performed using the previous regression equation (regression equation specified by t-1). The stop condition is not limited to these, and other conditions may be used.

[Configuration of prediction device]
Next, the configuration of the prediction apparatus 10 to which the proposed method of the present application is applied will be described. In addition, a present Example demonstrates the case where prediction of electric power demand is performed by the prediction apparatus 10. FIG. Further, in the present embodiment, a case will be described in which the prediction apparatus 10 performs prediction of power demand by specifying a regression equation from a plurality of explanatory variables using the above-described explanatory variable selection method.

  FIG. 3 is a diagram illustrating an example of a functional configuration of the prediction apparatus. As illustrated in FIG. 3, the prediction device 10 includes a display unit 20, an input unit 21, a storage unit 22, and a control unit 23. The prediction device 10 may have various known functional units in addition to the functional units illustrated in FIG. For example, the prediction device 10 may include a communication interface unit that performs communication with other terminals.

  The display unit 20 is a display device that displays various types of information. Examples of the display unit 20 include a display device such as an LCD (Liquid Crystal Display). The display unit 20 displays various information. For example, the display unit 20 displays various operation screens and prediction results.

  The input unit 21 is an input device that inputs various types of information. For example, examples of the input unit 21 include an input device such as a keyboard and a mouse connected to the prediction device 10, various buttons provided on the prediction device 10, and a transmissive touch sensor provided on the display unit 20. . In the example of FIG. 3, since the functional configuration is shown, the display unit 20 and the input unit 21 are separately provided. For example, the display unit 20 and the input unit 21 such as a touch panel are integrally configured. May be.

  The storage unit 22 is a storage device that stores various data. For example, the storage unit 22 is a storage device such as a hard disk, an SSD (Solid State Drive), or an optical disk. The storage unit 22 may be a semiconductor memory that can rewrite data such as a random access memory (RAM), a flash memory, and a non-volatile static random access memory (NVSRAM).

  The storage unit 22 stores an OS (Operating System) executed by the control unit 23 and various programs. For example, the storage unit 22 stores various programs including a prediction program that executes a prediction process described later. Furthermore, the storage unit 22 stores various data used in programs executed by the control unit 23. For example, the storage unit 22 stores learning data 30.

  The learning data 30 is data used for calculating optimum values of various parameters in the regression equation of ridge regression. The learning data 30 stores the actual value of the objective variable of the regression equation of ridge regression and the actual value of the explanatory variable when the actual value is obtained. In the present embodiment, the actual value of the power demand for a predetermined period and the actual values of a plurality of explanatory variables such as the temperature, humidity, and wind speed when the power demand is obtained are stored.

  The control unit 23 is a device that controls the prediction device 10. As the control unit 23, an electronic circuit such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array) can be employed. The control unit 23 has an internal memory for storing programs defining various processing procedures and control data, and executes various processes using these. The control unit 23 functions as various processing units by operating various programs. For example, the control unit 23 includes a reception unit 40, a specification unit 41, a generation unit 42, a calculation unit 43, a prediction unit 44, and an output control unit 45.

  The reception unit 40 performs various types of reception. For example, the reception unit 40 receives input of various information related to power demand prediction and various operation instructions related to power demand prediction. For example, the reception unit 40 displays an operation screen (not shown) on the display unit 20 and receives various operation instructions by an input operation from the input unit 21. For example, the reception unit 40 receives an instruction to start prediction of power demand.

  The specifying unit 41 specifies a regression equation that is effective for prediction of power demand by the above-described explanatory variable selection method. For example, based on the learning data 30, the specifying unit 41 sets the predicted value and the actual value of the objective variable calculated from the regression equation among a plurality of explanatory variables stored in the learning data 30 until a predetermined stop condition is satisfied. By repeating the selection of an explanatory variable having a large absolute value of the correlation with respect to the error, a regression equation using the selected explanatory variable is specified. For example, the specifying unit 41 first calculates the correlation with the actual value of each explanatory variable using the actual value of the power demand stored in the learning data 30 as an error as it is. And the specific | specification part 41 specifies the explanatory variable with the largest absolute value of the calculated correlation. The specifying unit 41 selects the explanatory variable having the maximum absolute value of the correlation. Thereafter, the specifying unit 41 calculates the predicted value of the power demand by substituting the actual result of the explanatory variable stored in the learning data 30 into the regression equation based on the selected explanatory variable, and calculates the predicted value of the power demand and the learned data 30. An error from the stored actual value is calculated. And the specific | specification part 41 calculates | requires the correlation with respect to an error about each explanatory variable not contained in a regression equation, selects an explanatory variable with a large absolute value of a correlation, and adds it to a regression equation. The specifying unit 41 repeats the selection of the explanatory variable having the maximum absolute value of the correlation until the error between the predicted value of power demand and the actual value calculated from the regression equation satisfies the stop condition in this way.

The generating unit 42 generates a plurality of candidate values for the conversion parameters of the regression equation to which the explanatory variable is added in the specifying unit 41. For example, the generation unit 42 generates the conversion parameter candidate value δ evenly in the range of 0 <δ ≦ 1, using the above-described formula (14) as the regularization parameter conversion formula. For example, the generation unit 42 generates 50 candidate values δ _i by changing D from 1 to D, assuming D = 50 in the equation (15).

  The calculating unit 43 calculates an estimated value of the prediction error of the regression equation when the candidate values generated by the generating unit 42 are applied to the conversion parameters to the regression equation to which the explanatory variable is added in the specifying unit 41, respectively. . For example, the calculation unit 43 converts the regularization parameter of the regression equation to which the explanatory variable is added in the specifying unit 41 using a conversion equation. Then, the calculation unit 43 sequentially applies the candidate value δ to each conversion parameter of the regression equation, and uses the AIC, BIC, cross-validation, Mallow's Cp, and the like to estimate the prediction error of the regression equation from the learning data 30. Is calculated. For each explanatory variable, the calculation unit 43 applies a candidate value that minimizes the estimated value of the calculated prediction error to the conversion formula to obtain a regularization parameter. Thereby, a regularization parameter with a small error is specified for each explanatory variable. The specifying unit 41 calculates an error for each explanatory variable by using a regression equation to which a regularization parameter with a small error is applied.

  That is, in the prediction device 10, the specifying unit 41 selects an explanatory variable effective for prediction, specifies a regression equation based on the selected explanatory variable, and generates a conversion parameter candidate value for the regression equation by the generating unit 42. Then, by calculating the estimated value of the prediction error of the regression equation when the candidate value is applied to the conversion parameter by the calculation unit 43, a regularization parameter that is configured with explanatory variables effective for prediction and that has a small error is applied. The regression equation of the obtained ridge regression is obtained.

  The prediction unit 44 performs prediction based on the obtained regression formula of ridge regression. In the present embodiment, the prediction unit 44 predicts power demand.

  The output control unit 45 controls various outputs. For example, the output control unit 45 causes the display unit 20 to display power demand information predicted by the prediction unit 44. Further, for example, the output control unit 45 outputs the information on the power demand predicted by the prediction unit 44 to an external terminal device. Thereby, the user can grasp | ascertain the prediction result of an electric power demand from the output information.

[Prediction example]
Here, an example of power demand prediction will be described. In addition, a present Example demonstrates the case where the electric power demand is estimated by calculating | requiring the annual cycle pattern of the electric power demand of 12:00 pm as a regression equation. Here, a Fourier series is used as a regression equation for ridge regression. The regression equation can be expressed as the following equation (22), for example.

  Here, y is an objective variable and is a predicted power demand at 12:00. It is assumed that t is a number from 1 to 365 assigned in order from January 1 to December 31. T represents a period and is 365. π is the circumference ratio. β is a regression coefficient.

  In the regression equation shown in the equation (22), the waveform when the variable l is 1 to 100 is weighted and added by the ratio of the regression coefficient. In this case, the explanatory variable is a 200-column matrix in which sin (2πtl / T) and cos (2πtl / T) are assigned to each column for each variable l, and the number of explanatory variables appearing in the regression equation is , 200.

  In the following example, a case where a regression coefficient of ridge regression is obtained using a method for normal ridge regression will be described. Note that a generalized method for ridge regression may be used.

  The learning data 30 stores the actual value of power demand at 12:00 pm during a predetermined period. For example, the learning data 30 stores the actual value of the power demand at 12:00 pm within TEPCO from September 30, 2012 to September 29, 2013. For example, the actual value of power demand is stored in the learning data 30 in association with the value of t assigned to January 1 to December 31.

  The prediction device 10 obtains a regression coefficient of ridge regression for the learning data 30 using Equation (7). In the present embodiment, the prediction device 10 generates candidate values by setting k in Expression (14) to 365 which is the number of data of the learning data 30, and D in Expression (15) as 100, and the prediction error of each candidate value is calculated. A regularization parameter is determined by calculating an estimated value. The estimated value of the prediction error is obtained by LOOCV (Leave-One-Out Cross-Validation) which is a kind of cross-validation. For example, some date data of the learning data 30 is used as learning data, and the remaining date data is used as verification data. The reason for this is that a part of the learning data 30 is used as learning data, the remaining data is used as verification data, the regularization parameter is determined from the learning data, and the determined regularization parameter is applied. This is because the estimated value of the prediction error is obtained from the verification data using the regression equation thus obtained, so that the accuracy of the prediction of the regression equation can be appropriately evaluated. The prediction device 10 determines the regularization parameter using the learning data, performs prediction on the verification data using a regression equation to which the determined regularization parameter is applied, compares the predicted value with the actual value, and calculates the prediction error. Calculate an estimate. When calculating the regression coefficient, the explanatory variable and the objective variable are standardized.

  In order to evaluate the prediction result, the actual value data of the power demand at 12:00 p.m. within TEPCO from September 30, 2013 to September 29, 2014 is used as test data. If the prediction device 10 predicts the power demand for the test data with the regression coefficient obtained from the learning data 30, the simulation assumes an actual prediction, and the error for the test data can be regarded as a prediction error.

  FIG. 4 is a diagram illustrating an example of evaluation of a prediction result. FIG. 4 shows an error with respect to the learning data 30, an estimated value of the prediction error, and an error with respect to the test data with respect to the regression coefficient of the ridge regression obtained with each candidate value of the regularization parameter. Each error is calculated by RMSE (Root Mean Squared Error). The horizontal axis in FIG. 4 represents the value of δ. The vertical axis of FIG. 4 represents the magnitude of various errors in ridge regression using the corresponding δ. As shown in FIG. 4, the error with respect to the learning data 30 decreases monotonously as δ increases. On the other hand, it can be seen that the estimated value of the prediction error has a downwardly convex shape. This is because when δ is in the vicinity of 1, the regression equation is too fit for the learning data 30, and the prediction result based on the regression equation is overfitting.

  On the other hand, there is a difference between the prediction error estimation value and the test data error. This is because the estimation error is included in the estimated value of the prediction error by LOOCV, and the learning data 30 and the test data are data of power demand in different years, and the nature of the power demand has changed over time. . That is, the property of the test data may change with respect to the learning data 30. Since the prediction device 10 obtains the regression equation from the learning data 30 before the property changes, the prediction result based on the regression equation does not necessarily have the smallest error with respect to the test data. However, the prediction device 10 can predict the test data with high accuracy by obtaining a regression equation that minimizes the estimated value of the prediction error for the learning data 30.

  The effectiveness of the method of the present embodiment depends on the conversion of equation (14). The property required for this conversion is that the prediction error does not change significantly in the vicinity of δ where the prediction error is minimized compared to the interval of candidate values of δ. Since the error curve for the test data shown in FIG. 4 is smooth, it is possible to confirm the effectiveness of the conversion of equation (14).

  In the example of FIG. 4, the value of δ is 0.86, and the estimated value of the prediction error is minimum. The prediction device 10 determines the regularization parameter by setting the value of δ to 0.86, and predicts the power demand by a regression equation to which the determined regularization parameter is applied. The error for the test data at δ = 0.86 is 3.81 million kW.

  Moreover, the prediction apparatus 10 may obtain the regression equation by selecting an explanatory variable using an explanatory variable selection method. For example, when sin (2πtl / T) and cos (2πtl / T) in the above equation (22) are used as explanatory variables, the number of explanatory variables appearing in the regression equation is 200. The prediction apparatus 10 may obtain a regression equation by selecting an effective explanatory variable from 200 explanatory variables using an explanatory variable selection method.

  For example, when the prediction device 10 selects 80 variables based on the explanatory variable selection method for the learning data 30 under the same conditions as when the above-described power demand is predicted, the obtained regression equation test The error for the data is 3.86 million kW. The error for the ridge regression test data using 200 explanatory variables is 3.81 million kW. Thus, it can be seen that the prediction device 10 obtained almost the same prediction performance even when the number of explanatory variables used was reduced to 80 using the explanatory variable selection method. Thus, the prediction apparatus 10 can select an explanatory variable that is important in prediction by using the explanatory variable selection method.

  In this way, the prediction device 10 generates a plurality of conversion parameter candidate values within a possible range of the conversion parameter of the conversion equation for converting the regularization parameter of the regression equation. Thus, the conversion parameter candidate value can be generated without trial and error by determining the range that the conversion parameter can take. Conventionally, since there is no limit on the upper limit that can be taken for the regularization parameter, it has been difficult to realize the automation of the process because it requires trial and error to determine the range of candidate values for the regularization parameter. On the other hand, in the prediction device 10, since the range that can be taken by the conversion parameter of the conversion equation for converting the regularization parameter is determined, it is possible to realize the automation of the processing for obtaining the regularization parameter of the regression equation. As a result, the prediction device 10 can efficiently search for an appropriate regularization parameter.

  Further, the prediction device 10 converts the regularization parameter of the regression equation into a conversion parameter using a conversion equation that approximately linearly converts the influence of the change of the conversion parameter on the prediction of the regularization parameter of the regression equation, Candidate values for a plurality of conversion parameters are generated at equal intervals over the range that the conversion parameters can take. Thereby, the prediction apparatus 10 can search for the candidates for the regularization parameter uniformly in the range that can be taken by the regularization parameter with the fineness according to the number of candidate values, and the accuracy according to the number of candidate values Can be searched for good regularization parameters. Further, the prediction device 10 can perform prediction with high accuracy by a regression equation to which the searched regularization parameter is applied. In addition, since the prediction apparatus 10 only needs to generate conversion parameter candidate values at equal intervals over the range that can be taken by the conversion parameter, it is possible to realize the automation of the process for obtaining the regularization parameter of the regression equation. As a result, the prediction device 10 can efficiently search for an appropriate regularization parameter.

[Process flow]
A flow of prediction processing in which the prediction device 10 according to the present embodiment performs prediction will be described. FIG. 5 is a flowchart illustrating an example of the procedure of the prediction process. This prediction process is executed at a predetermined timing, for example, at a timing when the reception unit 40 receives an instruction to start prediction of power demand.

As illustrated in FIG. 5, the specifying unit 41 initializes an area for selecting an explanatory variable (S10). For example, the generation unit 42 empties the area A ₀ = {} that holds the selected explanatory variable. The generation unit 42 initializes Y ^ ₀ = 0 and initializes the counter t = 1.

Specification unit 41, for each explanatory variable unselected other than those held in the area A _0, and calculates the correlation between the error of the regression equation by the selected explanatory variables (S11). Specification unit 41, first, A ₀ is empty, since there is no selected explanatory variables, as it is error actual value of the objective variable, for each explanatory variable, calculates a correlation between the error.

  The identifying unit 41 obtains an explanatory variable having the maximum absolute value of the correlation with the error calculated in S11 from the unselected explanatory variables (S12).

Specification unit 41, the absolute value of the correlation is to add up explanatory variables in the area A _t, the absolute value of the correlation incorporates a maximum of explanatory variables in the regression equation (S13).

The generation unit 42 converts the regularization parameters of the regression equation using the conversion equations, and generates the candidate value δ _i of the conversion parameter δ (S14). Based on the learning data 30, the calculation unit 43 converts the regularization parameter of the regression equation by the conversion equation, and calculates the estimated value of the prediction error of the regression equation when the candidate value is applied to each conversion parameter (S15).

  The specifying unit 41 applies the candidate value that minimizes the estimated value of the prediction error to the conversion parameter, obtains the regression coefficient for each explanatory variable, and specifies the regression equation (S16).

Specification unit 41, based on the training data 30 to determine the error of the predicted value and the actual value of the objective variable that is calculated regression coefficients beta _t from the applied regression equation (S17).

  The identifying unit 41 determines whether the error satisfies the stop condition (S18). If the stop condition is not satisfied (No at S18), the process proceeds to S11 described above.

  On the other hand, if the stop condition is satisfied (Yes at S18), the process is terminated.

[effect]
As described above, the prediction device 10 according to the present embodiment uses the actual value of the objective variable of the regression equation of the ridge regression for obtaining the objective variable from the value of the explanatory variable, and the explanatory variable when the actual value is obtained. The learning data 30 in which the actual value is stored is stored. The prediction apparatus 10 can obtain conversion parameter candidates in a predetermined range that can be taken by the conversion parameter of the conversion formula that can obtain a value in the range that can be taken by the regularization parameter of the regression formula by changing the value of the predetermined range of the predetermined conversion parameter Generate multiple values. Based on the learning data 30, the prediction device 10 converts the regularization parameter of the regression equation by the conversion equation, and calculates the estimated value of the prediction error of the regression equation when each of the generated candidate values is applied to the conversion parameter. To do. The prediction device 10 performs prediction using a regression equation in which a candidate value that minimizes the estimated value of the prediction error is applied to the conversion parameter. Thereby, the prediction apparatus 10 can search an appropriate regularization parameter efficiently.

  Further, in the prediction device 10 according to the present embodiment, the conversion formula approximately linearly converts the influence of the change of the conversion parameter on the prediction of the regularization parameter. The prediction device 10 generates candidate values for a plurality of conversion parameters at equal intervals within a predetermined range. As a result, the prediction device 10 can search for candidates for the regularization parameter uniformly over the possible range of the regularization parameter with a fineness according to the number of candidate values.

  Moreover, the prediction apparatus 10 which concerns on a present Example makes a conversion type | formula above-mentioned Formula (14). Thereby, the prediction device 10 can efficiently search for an appropriate regularization parameter, and can perform prediction with high accuracy by a regression equation to which the searched regularization parameter is applied.

  In addition, the prediction device 10 according to the present embodiment, based on the learning data 30, until the predetermined stop condition is satisfied, among the plurality of explanatory variables included in the regression equation, the predicted value of the objective variable calculated from the regression equation The regression equation based on the selected explanatory variable is specified by repeatedly selecting an explanatory variable having a large absolute value of the correlation with the error from the actual value. At that time, the prediction device 10 generates a plurality of conversion parameter candidate values for the regularization parameter of each explanatory variable of the identified regression equation. The prediction device 10 converts the regularization parameter of each explanatory variable of the identified regression equation by the conversion equation, and the prediction error of the regression equation when the plurality of generated candidate values are respectively applied to the conversion parameter. Calculate an estimate. Then, the prediction device 10 predicts the regularization parameter of each explanatory variable of the identified regression equation by the regression equation in which the candidate value that minimizes the estimated value of the calculated prediction error is applied to the conversion parameter. Thereby, the prediction apparatus 10 can select an explanatory variable important in prediction, and can perform prediction using a regression equation based on the selected explanatory variable. Moreover, since the prediction apparatus 10 selects an explanatory variable important in prediction and specifies a regression equation, the prediction processing load can be reduced.

  Further, the prediction device 10 according to the present embodiment is configured to select a predetermined stop condition, a predetermined number of explanatory variables having a high correlation, or a predicted value of an objective variable calculated from a regression equation based on the selected explanatory variable. Until the error with the actual value increases. When the predetermined stop condition is until a predetermined number of explanatory variables having a high correlation is selected, the number of times that the explanatory variable is selected is determined, so that the specific processing time of the regression equation can be prevented from becoming long. . Further, when the prediction apparatus 10 determines that the predetermined stop condition is until the error between the predicted value of the objective variable calculated from the regression equation based on the selected explanatory variable and the actual value increases, a regression equation with a small error is obtained. Specific predictions can be made.

  Moreover, the prediction apparatus 10 according to the present embodiment predicts power demand by a regression equation. Since the prediction device 10 can automate the process for obtaining the regularization parameter of the regression equation, the process for predicting the power demand can be automated.

  Although the embodiments related to the disclosed apparatus have been described so far, the disclosed technology may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

  For example, in the above-described embodiment, the case where the conversion formula is the above-described formula (14) has been described, but the disclosed apparatus is not limited thereto. For example, regularization parameter conversion represented by the following equation (23) may be used.

  Here, m is a positive real number.

  Moreover, although said Example demonstrated the case where electric power demand was estimated, the apparatus of an indication is not limited to this. Any ridge regression can be applied to any prediction.

  In the above-described embodiment, the case where an effective explanatory variable is selected by an explanatory variable selection method to obtain a regression equation has been described. However, the disclosed apparatus is not limited to this. The explanatory variable selection method may not be performed.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific state of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the processing units such as the reception unit 40, the specification unit 41, the generation unit 42, the calculation unit 43, the prediction unit 44, and the output control unit 45 may be appropriately integrated. Further, the processing of each processing unit may be appropriately separated into a plurality of processing units. Further, all or any part of each processing function performed in each processing unit can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. .

[Prediction program]
The various processes described in the above embodiments can also be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. Therefore, in the following, an example of a computer system that executes a program having the same function as in the above embodiment will be described. FIG. 6 is a diagram illustrating a computer that executes a prediction program.

  As illustrated in FIG. 6, the computer 300 includes a central processing unit (CPU) 310, a hard disk drive (HDD) 320, and a random access memory (RAM) 340. These units 300 to 340 are connected via a bus 400.

  The HDD 320 stores in advance a prediction program 320 a that exhibits the same functions as the reception unit 40, identification unit 41, generation unit 42, calculation unit 43, prediction unit 44, and output control unit 45. Note that the prediction program 320a may be separated as appropriate.

  The HDD 320 stores various information. For example, the HDD 320 stores various data used for prediction such as the learning data 30 described above.

  And CPU310 reads the prediction program 320a from HDD320, and performs the operation | movement similar to each process part of an Example. That is, the prediction program 320 a performs the same operations as the reception unit 40, the specification unit 41, the generation unit 42, the calculation unit 43, the prediction unit 44, and the output control unit 45.

  Note that the above-described prediction program 320a is not necessarily stored in the HDD 320 from the beginning.

  For example, the program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into the computer 300. Then, the computer 300 may read and execute the program from these.

  Furthermore, the program is stored in “another computer (or server)” connected to the computer 300 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 300 may read and execute the program from these.

DESCRIPTION OF SYMBOLS 10 Prediction apparatus 20 Display part 21 Input part 22 Storage part 23 Control part 30 Learning data 40 Reception part 41 Identification part 42 Generation part 43 Calculation part 44 Prediction part 45 Output control part

Claims (8)

A storage unit for storing the actual value of the objective variable of the regression equation of the ridge regression for obtaining the objective variable from the value of the explanatory variable, and learning data in which the actual value of the explanatory variable when the actual value is obtained;
When the regularization parameter is λ, the conversion parameter is δ, k is a positive real number, and m is a positive real number, the following equation (1) is a conversion equation, and the conversion parameter δ is 0 <δ ≦ 1 A generation unit that generates a plurality of conversion parameter candidate values within the range ,
Based on the learning data, the regularization parameter of the regression equation is converted by the conversion equation, and the estimated value of the prediction error of the regression equation when the plurality of candidate values generated by the generation unit are applied to the conversion parameters, respectively. A calculation unit for calculating
A prediction unit that performs prediction using a regression equation in which a candidate value that minimizes an estimated value of a prediction error calculated by the calculation unit is applied to a conversion parameter;
The prediction apparatus characterized by having.

Before Symbol conversion formula, prediction apparatus according to claim 1, characterized in that the formula (2) shown below.

The generator, forward prediction apparatus according to claim 2, characterized in that the Kihan circumference generates a candidate value of the plurality of conversion parameters at equal intervals. The regression equation of the ridge regression includes a plurality of explanatory variables,
Based on the learning data, until a predetermined stop condition is satisfied, an explanatory variable having a large absolute value of a correlation with respect to an error between the predicted value of the objective variable calculated from the regression equation and the actual value among the plurality of explanatory variables. It further includes a specifying unit that repeats the selection and specifies a regression equation based on the selected explanatory variable,
The generating unit generates a plurality of conversion parameter candidate values for the regularization parameter of each explanatory variable of the identified regression equation,
The calculation unit converts the regularization parameter of each explanatory variable of the identified regression equation by the conversion equation, and applies the plurality of candidate values generated by the generation unit to the conversion parameter, respectively. Calculate an estimate of the prediction error in the equation,
The prediction unit predicts a regularization parameter of each explanatory variable of the identified regression equation by a regression equation in which a candidate value with the smallest estimated value of the prediction error calculated by the calculation unit is applied to a conversion parameter. It performs. The prediction apparatus as described in any one of Claims 1-3 characterized by the above-mentioned. The predetermined stop condition shall be until a predetermined number of explanatory variables with high correlation are selected, or until the error between the predicted value and actual value of the objective variable calculated from the regression equation based on the selected explanatory variable increases. The prediction apparatus according to claim 4 . The learning data stores power demand for a predetermined period as objective variables, and actual values of a plurality of explanatory variables when the power demand is obtained,
The prediction unit, the prediction apparatus according to any one of claims 1-5, characterized in that to predict the power demand by the regression equation. When the regularization parameter of the regression equation of ridge regression for obtaining the objective variable from the value of the explanatory variable is λ, the conversion parameter is δ, k is a positive real number, and m is a positive real number, the following equation (3 ) As a conversion formula, and a plurality of conversion parameter candidate values are generated in a range where the conversion parameter δ is 0 <δ ≦ 1 ,
Based on the learning data storing the actual value of the objective variable of the regression equation and the actual value of the explanatory variable when the actual value was obtained, the regularization parameter of the regression equation is converted by the conversion equation and generated Calculating an estimated value of the prediction error of the regression equation when each of the plurality of candidate values is applied to the conversion parameter,
A prediction method characterized in that a computer executes a process of performing a prediction using a regression equation in which a candidate value that minimizes an estimated value of a calculated prediction error is applied to a conversion parameter.

When the regularization parameter of the regression equation of ridge regression for obtaining the objective variable from the value of the explanatory variable is λ, the conversion parameter is δ, k is a positive real number, and m is a positive real number, the following equation (4 ) As a conversion formula, and a plurality of conversion parameter candidate values are generated in a range where the conversion parameter δ is 0 <δ ≦ 1 ,
Based on the learning data storing the actual value of the objective variable of the regression equation and the actual value of the explanatory variable when the actual value was obtained, the regularization parameter of the regression equation is converted by the conversion equation and generated Calculating an estimated value of the prediction error of the regression equation when each of the plurality of candidate values is applied to the conversion parameter,
A prediction program characterized by causing a computer to execute a process of performing prediction using a regression equation in which a candidate value that minimizes an estimated value of a calculated prediction error is applied to a conversion parameter.

Images