JP7139625B2 - Factor analysis system, factor analysis method and program - Google Patents

Description

The present invention relates to a factor analysis system, factor analysis method and program.

In the recent electric power system reform, for example, the "planned value simultaneous equalization system" has become a topic. Accurate demand forecasting is important in the "simultaneous planned value system". However, conventional demand forecasting conducted by energy companies (electric power companies, etc.) incorporates fluctuation factors such as "weather information" and "calendar information" into the input data for power demand in their own area (territory). It is often operated using a model that predicts

However, with the implementation of the above-mentioned electric power system reform, in addition to the existing electric power companies, electric power retailers called "new electric power companies" (hereinafter referred to as new electric power companies) will participate as new energy companies. is possible.

The customer's power demand predicted by the new electric power company fluctuates greatly on a daily basis depending on the contract status. Therefore, the operator (hereafter referred to as the user) periodically analyzes the fluctuation factors of power demand faced by the new power company, and predicts the new power demand based on the input data that reflects the analysis results. necessary. Prediction of the new power demand is necessary not only for new power companies but also for existing power companies.

Correlation structure models that use correlation coefficients, which have been used in conventional electricity demand forecasting, can obtain pure relationships between explanatory variables and objective variables due to apparent correlations called "pseudo-correlations." Can not.

As a related technology, for maximum power forecasting, a technology has been proposed in which demand factors that are likely to be related to power demand are analyzed in advance by correlation analysis, and based on the results, input data to be used for power demand forecasting is selected. (See, for example, Non-Patent Document 1).

Correlation analysis is a method of quantitatively analyzing the relationship between factors in the range of -1 to 1 using correlation coefficients. However, in correlation analysis, if there are many factors to be analyzed, it may not be possible to determine up to which threshold the correlation coefficient of variables should be used as input data.

In recent years, a technique called graphical modeling using a partial correlation coefficient has been used for demand forecasting (see Patent Document 2, for example). A correlation structure model using graphical modeling can remove the above-described "pseudo-correlation", so it is possible to select appropriate factors when performing demand forecasting.

Graphical modeling assumes that the partial correlation coefficient between certain factors is 0 (uncorrelated) and estimates the other partial correlation coefficients by assuming conditional independence. By excluding factors with a number of 0 (no correlation) as input data for demand forecasting, the number of factors can be reduced and the factors can be narrowed down.

Takeshi Haida, Shoichi Mutoh, "Development of Maximum Prediction Support System Based on Multiple Regression Method", Operations Research, Vol.41, No.9, pp.476-480 (1996-9) Eitaro Kurata, Hiroyuki Mori, "Input variable selection method for short-term power load forecasting using graphical modeling", 2007 Institute of Electrical Engineers of Japan Division B Conference, No.49 (2007-9)

Both the correlation analysis and graphical modeling mentioned above are analytical methods based on statistics, and are effective analytical methods for linear factors. descend.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to improve the accuracy of factor analysis.

In order to solve the above problems, one aspect of the present invention is
an input processing unit that performs input processing on non-linear analysis target data related to factor analysis;
a data preprocessing unit that performs predetermined processing on the data to be analyzed;
a linear part extracting unit for extracting linear part data from the analysis target data input processed by the input processing unit or from the analysis target data processed by the data preprocessing unit;
a factor analysis unit that analyzes the relationship of factors with respect to the prediction target with respect to the data of the linear portion extracted by the linear portion extraction unit, and performs control for quantitatively displaying the analysis results;
and
The factor analysis unit selects variables based on analysis results obtained by applying a multivariate analysis method to the linear part extracted by the linear part extraction unit,
It is characterized by

In the above, the analysis target data is characterized by including at least one or more of prediction target information of an energy supplier, weather information, calendar information, and event information.

In the above, the data preprocessing unit is characterized by visualizing the relationship between the prediction target and the factors.

Further, in any one of the above, the data preprocessing unit is characterized by performing one or both of removing abnormal data and interpolating missing data included in the data to be analyzed.

In the above, the linear part extraction unit is characterized by extracting a linear part from the analysis object data using a decision tree or clustering method.

In the above, the factor analysis unit is characterized by applying graphical modeling as the multivariate analysis method.

In the above, the factor analysis unit divides a plurality of factors included in the analysis target data into a plurality of groups, performs the graphical modeling for each of the one or more factors belonging to the plurality of groups, and then integrates the results. characterized in that

In order to solve the above problems, another aspect of the present invention is
Perform input processing for non-linear analysis target data related to factor analysis,
performing predetermined processing on the data to be analyzed;
extracting data of a linear part from the data to be analyzed that has undergone the input process or from the data to be analyzed that has undergone the predetermined processing;
Analyze the relationship of factors with respect to the prediction target for the extracted data of the linear part, and perform control to quantitatively display the analysis results,
Analyzing the relationship of the factors includes performing variable selection based on analysis results obtained by applying a multivariate analysis method to the extracted linear portion.
It is characterized by

In order to solve the above problems, still another aspect of the present invention is
Perform input processing for non-linear analysis target data related to factor analysis,
performing predetermined processing on the data to be analyzed;
extracting data of a linear part from the data to be analyzed that has undergone the input process or from the data to be analyzed that has undergone the predetermined processing;
Analyzing the relationship of factors with respect to the prediction target for the extracted data of the linear portion, and performing control to quantitatively display the analysis results , wherein analyzing the relationship of the factors is the extraction performing variable selection based on analysis results obtained by applying a multivariate analysis technique to the linear part obtained ;
It is characterized by having a program for causing a computer to execute processing.

ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to improve the analysis precision at the time of factor analysis.

It is a figure showing an example of hardware constitutions of a factor analysis system concerning an embodiment of the present invention. It is a figure which shows an example of the data structure of the analysis object data in the input part of the factor-analysis system which concerns on embodiment of this invention. It is a figure which shows an example of the hardware constitutions of a general computer. FIG. 5 is a diagram showing a specific example of data visualization processing and data removal/interpolation processing according to the embodiment of the present invention; It is a figure which shows an example of data processing in the data pre-processing part which concerns on embodiment of this invention. It is a figure which shows an example of the decision tree in the linear part extraction part which concerns on embodiment of this invention. It is a figure which shows an example of linear part extraction by the clustering method based on embodiment of this invention. It is a figure which shows the example of an analysis result in which the quantitative index based on the factor-analysis part of the factor-analysis system which concerns on embodiment of this invention was shown. It is a figure which shows an example of the report based on the analysis result by the factor-analysis part of the factor-analysis system which concerns on embodiment of this invention. It is a flowchart explaining operation|movement of the factor-analysis part of the factor-analysis system which concerns on embodiment of this invention. FIG. 10 is an image diagram showing a process of dividing data to be analyzed by a factor analysis unit in the factor analysis system according to the embodiment of the present invention; FIG. 12 is a diagram showing a specific example of data before division of the analysis target data shown in FIG. 11; FIG. 12 is a diagram showing a specific example of divided data 1 after dividing the analysis target data shown in FIG. 11; FIG. 12 is a diagram showing a specific example of split data 2 after splitting the analysis target data shown in FIG. 11; It is an image figure which shows the state of integration of the analysis result of the factor-analysis part in the factor-analysis system which concerns on embodiment of this invention. 12C is a diagram showing a specific example of analysis results corresponding to the divided data 1 before integration shown in FIG. 12B; FIG. FIG. 12C is a diagram showing a specific example of analysis results corresponding to divided data 2 before integration shown in FIG. 12C. 14B is a diagram showing a specific example of an analysis result obtained by integrating the analysis result of the divided data 1 shown in FIG. 14A and the analysis result of the divided data 2 shown in FIG. 14B. FIG. 4 is a flow chart illustrating an algorithm in graphical modeling according to an embodiment of the invention; Fig. 4 is a flow chart illustrating an algorithm in a decision tree according to an embodiment of the invention; It is a flowchart explaining operation|movement of the factor-analysis part of the factor-analysis system which concerns on another embodiment of this invention. FIG. 11 is an image diagram showing how a factor analysis unit removes discrete value factors from analysis target data in a factor analysis system according to another embodiment of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following embodiments, the factor analysis system will be described as a system that analyzes fluctuation factors when forecasting power demand. However, the factor analysis system may be used to analyze various factors other than fluctuation factors when predicting power demand.

Since the maximum power demand fluctuates due to various factors such as holidays and weather conditions, the data to be analyzed by the factor analysis system of the embodiment (hereinafter referred to as analysis target data) is non-linear in nature. is normal. Therefore, the analysis target data in the embodiment will be described as non-linear data. Therefore, when analyzing data to be analyzed, it is not possible to apply the linear method as in the prior art as it is.

FIG. 1 is a diagram showing an example of the hardware configuration of a factor analysis system according to an embodiment of the invention. The hardware of the factor analysis system 100 is represented by functional blocks, and includes an input unit 10 for inputting data contributing to the analysis target (the above-described analysis target data), the input analysis target data divided into variable items, and stored in the storage device 30. Arithmetic device 20 that selects and processes data of variable items that are stored and retrieved from storage device 30, storage device 30 that stores input analysis object data by dividing it into variable items, and data output from computation device 20. and an output data storage/output unit 40 for storing output data.

A display or the like may be applied as the output data storage/output unit 40 that displays the output data. Also, the data displayed on the display or the like may be modifiable, for example, based on a GUI (Graphical User-Interface).

Next, the factor analysis system shown in FIG. 1 will be described in more detail. The input unit 10 transmits data (analysis target data) that can be analyzed, such as demand information under the jurisdiction of energy companies, weather information and calendar information provided by the Meteorological Agency, etc., and event information planned by energy companies, to the arithmetic device 20. input. The input unit 10 may be a keyboard or the like. In addition, the input unit 10 may input analysis target data acquired from a device such as a LAN (Local Area Network) (not shown) or a measurement sensor to the arithmetic device 20 . The computing device 20 that has received the data to be analyzed divides the data into variable items (eg, demand information, weather information, calendar information, event information, etc.) and stores them in the storage device 30 based on the data to be analyzed.

Then, the arithmetic device 20 retrieves the stored data from the storage device 30, sets objective variable data and explanatory variable data as data to be analyzed, and performs factor analysis.

The data to be analyzed includes whether or not it will be used as input data for demand forecasts, data on past actual values or forecast values to be analyzed, qualitative variables (e.g., day of the week information) that are nominal scales representing differences in categories, and dummy variables. Discrete values (0,1) etc. can be used.

By the way, data to be analyzed includes demand (objective variable data) and factors (explanatory variable data). The factor is a demand related factor. For example, the analysis target data is stored in the storage device 30 as the data structure shown in FIG. Storage device 30 may be a database.

The data to be analyzed in the example in Fig. 2 is from July 1, 2015 to March 31, 2016 as the data acquisition period, the maximum power demand of the day (Y1) as the objective variable data, and the previous day's maximum power demand as the explanatory variable data. Includes demand (X1), average temperature of the day (X2), solar radiation at 13:00 on the day (X3), events on the day (X4), etc. Data to be analyzed is not limited to the example in FIG.

For example, in the example of FIG. 2, the number of explanatory variable data (the number of factors) may be three or less, or may be five or more. In the example of FIG. 2, explanatory variable data further includes other factors (Xn: n is an integer of 5 or more).

As described above, the computing device 20 according to the embodiment of the present invention not only extracts data at a specified time from the analysis target data stored in the storage device 30, but also calculates statistical values such as maximum power demand and average temperature. In some cases, a factor analysis is performed by performing a process to do so.

The arithmetic unit 20 shown in FIG. 1 includes an input/selection processing unit 21, a data preprocessing unit 22, a linear part extraction unit 23, and a factor analysis unit . The function of each unit of the arithmetic device 20 may be implemented by, for example, a computer executing a predetermined control program.

The storage device 30 stores the analysis target data input via the input unit 10 as a database. The storage device 30 may store data other than those described above and intermediate calculation results.

The output data storage/output unit 40 has a function of storing quantitative analysis results analyzed by the factor analysis unit 24 and a function of outputting the results to a display or the like for display to the user.

The data stored in the output data storage/output unit 40 is periodically analyzed by changing only the analysis target period according to the user's settings. It is also possible to display the

FIG. 9 is a diagram showing an example of a report based on analysis results by the factor analysis unit 24 of the factor analysis system according to the embodiment of the present invention. In the example of FIG. 9, the analysis results for July-August 2015 and the analysis results for September-October 2015 are shown as bar graphs.

In the report of FIG. 9, the vertical axis indicates the quantitative index, and the horizontal axis indicates explanatory variable data. In the example of Figure 9, the analysis results for September and October 2015 are important variables (for example, the average temperature of the day (X2), the amount of solar radiation at 13:00 (X3 ), that day's event (X4), . . . ) has changed. The analysis result can present the user with materials for determining the necessity of changing the input data used for the factor analysis.

As described above, the arithmetic unit 20 is composed of general computer hardware, and the data taken out from the storage device 30 is processed according to the processing flow of the arithmetic unit 20 (see the arrow line in FIG. 1). A series of arithmetic processing is executed at the end, and the data output from the arithmetic processing at the final stage is stored and displayed in the output data storage/output unit 40 .

FIG. 3 is a diagram showing an example of the hardware configuration of a general computer. A computer 300 is connected via a bus 310 to a CPU (Central Processing Unit) 302 , a memory 304 , an input device 306 , an output device 308 , an external storage device 312 , a media drive device 314 , a network connection device 318 and the like. Note that the configuration of the computer 300 is applied to the arithmetic device 20 of this embodiment.

The CPU 302 is an arithmetic processing unit that controls the operation of the computer 300 as a whole. The memory 304 is a storage unit for pre-storing a program for controlling the operation of the computer 300 and for using it as a work area as necessary when executing the program. The memory 304 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), or the like.

The input device 306 is a device that, when operated by a user, acquires input of various types of information from the user associated with the content of the operation and sends the acquired input information to the CPU 302. For example, a keyboard device and a mouse. equipment and the like.

The output device 308 is a device that outputs the processing result of the computer 300, and includes a display device and the like. For example, the display device displays text and images according to display data sent by the CPU 302 .

The external storage device 312 is, for example, a storage device such as a hard disk, and is a device that stores various control programs executed by the CPU 302, acquired data, and the like.

A media drive device 314 is a device for writing to and reading from a portable recording medium 316 . CPU 302 can perform various control processes by reading and executing a predetermined control program recorded on portable recording medium 316 via medium drive device 314 .

The portable recording medium 316 includes, for example, CD (Compact Disc)-ROM, DVD (Digital Versatile Disc), USB (Universal Serial Bus) memory, and the like.

The network connection device 318 is an interface device that manages transmission and reception of various data to and from the outside by wire or wireless. Note that the input unit 10 of the present embodiment may communicate with the network connection device 318 described above. A bus 310 is a communication path that connects the devices and the like to exchange data.

Returning to FIG. 1, the arithmetic device 20 will be further described. The input/selection processing unit 21 can store the analysis target data acquired from the input unit 10 in the storage device 30 via the arithmetic device 20, or the data preprocessing unit without storing the analysis target data in the storage device 30. 22 can also be handed over.

The input/selection processing unit 21 selects objective variable data or explanatory variable data from the data to be analyzed taken out from the storage device 30, and transfers the data to the data preprocessing unit 22, which will be described later. The input/selection processing unit 21 functions as an input processing unit.

The data preprocessing unit 22 displays the analysis target data stored in the storage device 30 via the input/selection processing unit 21 on the display device (for example, the output device 308) of the arithmetic device 20, and calculates the demand (objective variable data). and factors (explanatory variable data) are visualized, and arithmetic processing is performed to remove abnormal data and interpolate missing data.

FIG. 4 is a diagram showing a specific example of data visualization processing and data removal/interpolation processing according to the embodiment of the present invention.

As described above, in the data visualization process executed by the data preprocessing unit 22, the relationship between the prediction target and the factors is displayed on the display device of the arithmetic unit 20 as a scatter diagram, a time series diagram, or the like, and a user (operator) GUI Based on the operation, the arithmetic device 20 can be accessed.

When the data preprocessing unit 22 recognizes a GUI operation for removing abnormal values, interpolating missing data, etc., for data displayed in a scatter diagram, a time-series diagram, or the like in the data visualization process, an abnormality is detected. Perform processing such as removing values and interpolating missing data. The data preprocessing unit 22 may automatically perform the processing.

Specifically, in the example of the scatter diagram in Fig. 4, the maximum power demand of the day (Y1) is plotted on the vertical axis, and the maximum power demand of the previous day (X1) is displayed on the screen on the horizontal axis. Data that greatly deviate from the above can be recognized as "abnormal values", and data are skipped (missing data) even though the data originally exists. Therefore, when the data preprocessing unit 22 recognizes a GUI operation for removing abnormal values, interpolating missing data, or the like, the data preprocessing unit 22 performs processing according to the recognized operation (processing for removing abnormal values, interpolating missing data, etc.). do

In the example of the time-series diagram in Fig. 4, the maximum power demand (Y1) of the day is displayed on the vertical axis, and the time-series date and time are displayed on the horizontal axis. If deviated data is displayed, it can be recognized as an "abnormal value", and it can be recognized that the data is discontinuous (missing data) even though the data originally exists. In this case, the data preprocessing unit 22 removes abnormal values, interpolates data, and the like based on the GUI operation described above or automatically.

FIG. 5 is a diagram showing an example of data processing in the data preprocessing unit according to the embodiment of the present invention. Comparing FIG. 2 and FIG. 5, since data processing by the data preprocessing unit 22 has not yet been performed in FIG. 2, abnormal values and missing data are included in the records in the table. Since the data has been processed by the data preprocessing unit 22, abnormal values and missing data in the table have been corrected (see footnotes (1) to (3) in FIG. 5).

The removal of data indicating anomalies or the interpolation processing of data is performed by the data preprocessing unit 22 based on statistical processing such as upper/lower limit filters, data verification, and spline interpolation. Removal of data determined to be present as well as interpolation of data can also be performed automatically. In this case, since the above processing is automatically performed, no GUI operation by the user is required.

Returning to FIG. 1, the linear part extractor 23 in the arithmetic unit 20 will be described. The linear part extraction unit 23 extracts a linear part from the analysis target data processed by the data preprocessing unit 22 . In this embodiment, the linear part extraction unit 23 applies a technique such as a decision tree or clustering to extract a linear part from the analysis target data, which is nonlinear data. FIG. 6 is a diagram showing an example of a decision tree in the linear partial extraction unit according to the embodiment of the present invention.

When the extraction process by the decision tree is started in FIG. 6, the linear part can be extracted from the non-linear original data shown on the right shoulder of the rhombus indicating the branch node by the if-then rule at the branch node. When a linear portion is extracted, rectangular terminal nodes such as linear portion 1 and linear portion 2 are formed and output as shown in FIG.

As a result, it is possible to automatically extract the data representing the linear portions as indicated by rectangular terminal nodes, that is, linear portion 1 and linear portion 2. FIG.

In this embodiment, the linear data shown in the linear part 1 and the linear part 2 are extracted from the non-linear original data shown in the right shoulder of FIG. Linear parts can be extracted by applying if-then rules to extracted branch nodes, even if they differ from those shown in the right shoulder.

Next, FIG. 7 shows an example of linear partial extraction by the clustering method according to the embodiment of the present invention. As shown in FIG. 7, by dividing the original non-linear data into an arbitrary number of clusters (in this embodiment, it is divided into two clusters 1 and 2), the data of the linear part is automatically extracted. can do.

The number of clusters may be set by the user using a GUI for the nonlinear data displayed on the screen, or may be automatically determined using an information criterion such as AIC (Akaike's Information Criterion). Also, in order to extract the linear portion with high accuracy, the distance function may be determined using the "Mahalanobis distance".

The linear part extraction unit 23 may acquire analysis target data from the input/selection processing unit 21 . In this case, the data to be analyzed has not been processed by the data preprocessing unit 22 (removal of data indicating abnormality, interpolation of data, etc.).

Therefore, it is preferable that the linear part extracting unit 23 extracts the linear part from the analysis target data processed as described above. However, the linear part extracting unit 23 can also extract the linear part from the analysis target data that has not been processed as described above.

Returning to FIG. 1 again, the factor analysis unit 24 in the arithmetic device 20 will be described. The factor analysis unit 24 quantitatively performs factor analysis considering factor selection using a multivariate analysis method for each linear part extracted by the linear part extraction unit 23, and displays the result on the data storage/output unit 40. do. The user can freely set which of the multivariate analysis methods to use.

FIG. 8 is a diagram showing an analysis result example showing quantitative indicators based on the factor analysis unit 24 of the factor analysis system according to the embodiment of the present invention.

In the example shown in Fig. 8, the explanatory variable data for July-8 Quantitative indices were calculated for each month, September to October, and the analysis results were output. FIG. 9 is a report showing the analysis results of the example of FIG. 8 in bar graph form. The report is presented to the user through screen display or the like, as described above.

The computing device 20 may not only show the analysis result of the explanatory variable data to the user based on the quantitative index, but may also display the analysis result information using a bar graph, a line graph, or the like. By displaying a bar graph, a line graph, or the like, the analysis results can be presented to the user in an easy-to-understand manner.

Further, when a plurality of linear parts are extracted by the linear part extracting unit 23, the analysis result of each result may be displayed as one result by statistical processing such as averaging or weighted averaging. The detailed method of factor analysis will be touched on later.

FIG. 10 is a flowchart for explaining the operation of the factor analysis section 24 of the factor analysis system according to the embodiment of the present invention. The factor analysis unit 24 includes processes of (1) division of data to be analyzed, (2) application of an analysis method considering factor selection, and (3) integration of analysis results. Each process will be described in order below with reference to FIG.

[1] Division of data to be analyzed (step S11)
In order to reduce the factors to be analyzed, the factors of the data to be analyzed (see FIG. 12A) are divided by an arbitrary number. Assuming that the number of explanatory variable data (factors) of the original data to be analyzed is T, the data to be analyzed is divided by an arbitrary number B of divisions. At this time, the objective variable of the divided analysis object data is always the same.

FIG. 11 is an image diagram showing the process of dividing the analysis target data of the factor analysis unit in the factor analysis system according to the embodiment of the present invention. In the original data to be analyzed shown on the left side of FIG. 11, the objective variable is Y1 only, and the explanatory variables (factors) are X1 . . . XT.

In the divided data 1 shown in the center, the objective variable Y1 is the same, and the explanatory variables (factors) are divided into X1 ... XB. factor) is divided into XB+1...X2B,... Note that the data acquisition period N is the same for all data during the division process.

12A is a diagram showing a specific example of data before division of the analysis target data shown in FIG. 11. FIG. That is, in FIG. 12A, the data acquisition period is from July 2015 to the end of October 2015, the maximum power demand of the day (Y1) is the objective variable data, the maximum power demand of the previous day (X1) is the explanatory variable data, and the average An example of analysis target data before division in which numerical values are specifically embedded is shown for the temperature (X2), the amount of solar radiation at 13:00 on the day (X3), and the event on the day (X4).

In the following description, an example will be described in which the factor analysis unit 24 divides the analysis target data shown in FIG. 12A such that T and B are "T/B=2". Therefore, the number of explanatory variable data (the number of factors) belonging to the divided group (divided data) is two. The number of explanatory variable data belonging to the group after division may be one.

FIG. 12B is a diagram showing a specific example of split data 1 after splitting the analysis target data shown in FIG. Divided data 1 after division shown in the example of FIG. 12B includes two explanatory variable data (the previous day's maximum power demand (X1) and the day's average temperature (X2)).

FIG. 12C is a diagram showing a specific example of divided data 2 after the analysis target data shown in FIG. 11 is divided. Divided data 2 after division shown in the example of FIG. 12C includes two explanatory variable data (insolation at 13:00 on the current day (X3) and event on the current day (X4)).

[2] Application of analysis method considering factor selection (step S13)
Returning to FIG. 10, in the application of the analysis method considering factor selection (step S13), the analysis method considering factor selection is applied to each data divided in step S11 until the set number of divisions is reached, and factor analysis is performed. do

[3] Integration of analysis results (step S14)
In integration of analysis results (step S14) shown in FIG. 10, a plurality of analysis results calculated in step S13 using the analysis method considering factor selection are integrated into one.

FIG. 13 is an image diagram showing how the analysis results of the factor analysis unit in the factor analysis system according to the embodiment of the present invention are integrated. As shown in FIG. 13, the output result of the divided data 1 divided in FIG. 12B and the output result of the divided data 2 divided in FIG. Even if the format is almost the same as the structure of the explanatory variable data in the data to be analyzed, it is possible to obtain analysis results with new quantitative indicators.

FIG. 14A is a diagram showing a specific example of the analysis result corresponding to the divided data 1 before integration shown in FIG. 12B, and is an embodiment of the image shown on the left end of FIG.

FIG. 14B is a diagram showing a specific example of the analysis result corresponding to the divided data 2 before integration shown in FIG. 12C, and is an embodiment of the image shown in the center of FIG.

FIG. 14C is a diagram showing a specific example of the analysis result obtained by integrating the analysis result of the divided data 1 shown in FIG. 14A and the analysis result of the divided data 2 shown in FIG. 14B. It embodies an image.

As described above, the factor analysis unit 24 performs factor analysis using multivariate analysis. One of the multivariate analysis methods is graphical modeling. When the factor analysis unit 24 applies graphical modeling to perform factor analysis, as the number of factors to be analyzed (the number of explanatory variable data) increases, the amount of calculation becomes enormous. For example, in the case of graphical modeling, the amount of computation is squared as the number of factors increases by one.

Therefore, the factor analysis unit 24 divides the explanatory variable data of the analysis target data as described above, thereby reducing the amount of calculation when performing the factor analysis. Therefore, even if the number of explanatory variable data increases, factor analysis can be performed with an appropriate amount of calculation.

As described above, for example, the linear part extraction unit 23 extracts a linear part from the nonlinear analysis target data, and the factor analysis unit 24 applies graphical modeling or the like to the extracted linear part, Conduct factor analysis. Therefore, analysis accuracy is improved.

In addition, graphical modeling assumes that the partial correlation coefficient between certain factors is 0 (uncorrelated) and estimates the other partial correlation coefficients by assuming conditional independence. By excluding 0 (uncorrelated) factors as input data for demand forecast, the number of factors can be reduced and the factors can be narrowed down.

It has already been described that the factor analysis unit 24 according to the embodiment of the present invention quantitatively performs factor analysis considering factor selection using a multivariate analysis method for each linear data extracted by the linear part extraction unit 23. As described above, an example of a multivariate analysis method considering factor selection in the factor analysis unit 24 will be described here.

The above-described graphical modeling will be described below as one method of multivariate analysis. Methods such as principal component analysis and multiple regression analysis may be applied to the multivariate analysis method, for example.

Graphical modeling is used in fields such as speech recognition, image processing, and marketing research as a method of graphically expressing partial correlation coefficients from which pseudo-correlation is removed between variables.

FIG. 15 is a flow chart explaining a graphical modeling algorithm in the factor analysis system according to the embodiment of the present invention.

In FIG. 15, the graphical modeling algorithm is divided into Steps 21 to 23 for explanation. i.e.
Step 21: Calculation of Partial Correlation Coefficient Matrix The objective variable data and explanatory variable data are treated as one matrix of learning data, and the partial correlation coefficient matrix P is calculated from the correlation coefficient matrix R of that matrix. A partial correlation coefficient is a correlation coefficient obtained by removing the influence of other related variables from the correlation between two variables. A formula for calculating a generalized partial correlation coefficient is shown in the following formula (1).

Step 22: Estimation of partial correlation coefficient matrix by covariance selection The one with the smallest absolute value in the partial correlation coefficient matrix is assumed to be conditionally independent (i,j), and the correlation coefficient matrix is updated by the following equation (2). do. Let S be the correlation coefficient matrix, and estimate the conditionally independent correlation coefficient matrix M by using the formula of the partitioned inverse matrix from Dempster's theorem.

A partial correlation coefficient matrix is calculated from the estimated correlation coefficient matrix M. When there are multiple conditions of independence, the coefficients that were previously conditionally independent are almost always non-zero, so we repeat this assumption iteratively until all of the chosen conditions of independence converge to zero. By repeating this, it is possible to set a convergence criterion that can be regarded as 0.

Step 23: Model Evaluation Since the criterion for covariance selection is determined by model evaluation using the Akaike Information Criterion (AIC), the covariance selection criterion is set when the AIC is the minimum for the goodness of fit of the model. Return to Step21.

Next, a decision tree as another method of multivariate analysis will be described. FIG. 16 is a flow chart explaining the decision tree algorithm in the factor analysis system according to the embodiment of the present invention.

A decision tree is a data mining method that is a methodology for extracting useful information, knowledge, and rules hidden in a large amount of data, and expresses input/output relationships in a tree structure based on if-then rules. is. An if-then rule is generally defined as a rule consisting of an if part representing a premise or condition and a then part representing a conclusion or action to be taken if the if part is true.

The decision tree algorithm shown in FIG. 16 is divided into Steps 31 to 34 and explained. i.e.
Step 31: Tree Growing Tree growing grows the tree by splitting the data in the parent node into two child nodes. First, a tree is constructed by selecting a branching condition that minimizes the error that occurs with respect to the data of the parent node that is the target data for the variable that is the factor. The degree of improvement of all input variables is calculated by the following equation (4), and the largest value among them is taken as the best branching condition. The input variable at that time is used as a branch input variable, and the divided right and left averages are used as branch values. By repeating this operation, the decision tree is grown to the maximum tree that cannot be subdivided any more.

Step32: Tree pruning In order to simplify the tree structure, pruning the branches of the tree once it has grown to the maximum size. At each branch node, find the error per number of nodes in the subtree below that node. Next, among the obtained values, the branch node with the smallest value is replaced with the terminal node. Finally, repeat until all branch nodes become terminal nodes. The maximum tree is once pruned to the minimum tree by the following procedure. The branch node error is defined as a complexity parameter in the following equation (5).

Step 33: Selection of Best Tree In the process of pruning trees, CART (Classification And Regression Trees) uses cross-validation as a decision tree error estimation method. The cross-validation method is a learning method for reducing learning bias or when learning data is insufficient during model building. First, the learning data is divided into ν groups, (ν−1) groups among them are used as learning data for model construction, and the remaining one group is used as test data for error estimation. The following equation (6) shows the error between the cross-validation method and the test data.

The post-pruning error is obtained by using the cross-validation method for each pruning according to the above equation (6). CART selects the best tree by using the best tree selection rule on the cross-validation error. The SE rule used in CART is shown in the following equation (7). A decision tree with the smallest number of nodes among the best candidate trees obtained by the SE rule is taken as the best tree.

Step 34: Calculation of variable importance The variable importance is an index that clarifies the degree of input variables when constructing a decision tree. Use the improvement of the variable used for the branch node in the best tree. The variable importance is a value obtained by summing this for each variable, and is shown in the following equation (8). The variable importance can quantitatively express the importance of the other variables, with the variable that is the most important for the prediction target set to 100.

The present invention is not limited to the above embodiments, and various improvements and modifications are possible without departing from the gist of the present invention. For example, the above-described embodiments may be improved and changed as follows.

In the above-described embodiment, the analysis target data acquired by the arithmetic device 20 via the input unit 10 or the storage device 30 is non-linear data, and may include discrete explanatory variable data (factor data). Discrete value factor data refers to data in which the number of items of the factor or the data value of the factor has less variation than the number of data of the factor. For example, calendar data representing weekdays as 0 and holidays as 1 has 2 items, and the number of items can be smaller than the number of data items. . On the other hand, for example, in temperature data, each data value corresponding to an item may be different, and the number of items may increase in proportion to the number of data. Therefore, temperature data may not correspond to discrete factor data. . The linear part extracting unit 23 uses a non-linear technique such as decision tree or clustering to extract a linear part from the data to be analyzed that may include the features described above. Then, the factor analysis unit 24 performs factor analysis on the extracted linear part using linear techniques such as graphical modeling and correlation analysis.

Certain factors may be correlated with other factors whose data may be discrete values. Therefore, in order to appropriately extract the linear part, the linear part extracting unit 23 preferably extracts the linear part from the analysis object data including the factor data that are discrete values. On the other hand, if the data to be analyzed includes discrete values in the linear method used by the factor analysis unit 24 for factor analysis, there is a possibility that an accurate analysis result cannot be obtained. Therefore, in another embodiment, the factor analysis unit 24 may perform factor analysis after removing factor data of discrete values from the linear portion that is the data to be analyzed.

Specifically, as shown in FIG. 17, the factor analysis unit 24 removes discrete value factor data from the linear part that is the analysis target data (Step S11) prior to the process of dividing the analysis target data (Step S11). S11´). FIG. 17 is a flowchart explaining the operation of the factor analysis unit of the factor analysis system according to another embodiment of the present invention.

In Step S11', the factor analysis unit 24 calculates a discrete value ratio for each explanatory variable (factor) of the data to be analyzed, which is a linear part. Then, the factor analysis unit 24 removes factor data, which are discrete values, from the analysis target data by comparing the calculated discrete value ratio with a preset threshold value.

For example, the factor analysis unit 24 calculates the discrete value ratio by dividing the number of items of the factor by the number of data of the factor, as shown in the following equation (9). Alternatively, the factor analysis unit 24 calculates the discrete value ratio by dividing the value obtained by subtracting the number of items of the factor from the number of data of the factor by the number of data of the factor, as shown in the following equation (10). do.

When Equation (9) is used to calculate the discrete value ratio, the factor analysis unit 24 removes factor data for which the calculated discrete value ratio is below the threshold from the analysis target data. Further, when Expression (10) is used to calculate the discrete value ratio, the factor analysis unit 24 removes factor data whose calculated discrete value ratio exceeds the threshold from the analysis target data. For example, as shown in FIG. 18, the factor analysis unit 24 removes the data of factor X2, which is a binary discrete value, and the data of factor X4, which is a ternary discrete value, from the data to be analyzed. FIG. 18 is an image diagram showing how a factor analysis unit removes discrete value factors from analysis object data in a factor analysis system according to another embodiment of the present invention.

The processing after Step S12 is the same as the processing described above. The factor analysis unit 24 outputs analysis results that do not include factors that are discrete values to the output data storage/output unit 40, and the output data storage/output unit 40 stores the analysis results and displays the analysis results on a display or the like. Output and display to the user.

As described above, in another embodiment, the factor analysis unit 24 performs factor analysis on the analysis target data, which is a linear part, using a linear technique such as graphical modeling or correlation analysis. Since discrete value factor data is removed from , factor analysis can be performed more accurately.

Further, the factor analysis unit 24 extracts the analysis result obtained for the factor having a discrete value when the linear part extracting unit 23 extracts the linear part of the data to be analyzed, together with the analysis result not including the factor having a discrete value. It may be output to the output data storage/output unit 40 . An example of the analysis result obtained by the linear part extraction unit 23 is the degree of influence given to the construction of a decision tree for factors that are not discrete values. The output data storage/output unit 40 may store the analysis results input from the factor analysis unit 24 and output the analysis results to a display or the like for the user. According to such a configuration, the user can also recognize the influence of factors that are discrete values on the demand (objective variable data).

REFERENCE SIGNS LIST 10 input unit 20 arithmetic unit 21 input/selection processing unit 22 data preprocessing unit 23 linear part extraction unit 24 factor analysis unit 30 storage device 40 output data storage/output unit 100 factor analysis system

Claims (11)

an input processing unit that performs input processing on non-linear analysis target data related to factor analysis;
a data preprocessing unit that performs predetermined processing on the data to be analyzed;
a linear part extracting unit for extracting linear part data from the analysis target data input processed by the input processing unit or from the analysis target data processed by the data preprocessing unit;
a factor analysis unit that analyzes the relationship of factors with respect to the prediction target with respect to the data of the linear portion extracted by the linear portion extraction unit, and performs control for quantitatively displaying the analysis results;
with
The factor analysis unit selects variables based on analysis results obtained by applying a multivariate analysis method to the linear part extracted by the linear part extraction unit,
A factor analysis system characterized by: In the factor analysis system according to claim 1,
A factor analysis system, wherein the analysis target data includes at least one or more of prediction target information of an energy supplier, weather information, calendar information, and event information. In the factor analysis system according to claim 1,
The factor analysis system, wherein the data preprocessing unit visualizes a relationship between a prediction target and factors. In the factor analysis system according to any one of claims 1 to 3,
The factor analysis system, wherein the data preprocessing unit performs one or both of removal of abnormal data and interpolation of missing data included in the data to be analyzed. In the factor analysis system according to claim 1,
The factor analysis system, wherein the linear part extracting unit extracts the linear part from the data to be analyzed using a decision tree or a clustering method. In the factor analysis system according to claim 1,
The factor analysis system, wherein the factor analysis unit removes factor data, which are discrete values, from the linear part extracted by the linear part extraction part. In the factor analysis system according to claim 6,
The factor analysis unit analyzes the factor that is a discrete value when the linear part extraction unit extracts the linear part together with the analysis result of the linear part from which the factor data that is a discrete value is removed. A factor analysis system characterized by outputting results. In the factor analysis system according to claim 1 ,
The factor analysis system, wherein the factor analysis unit applies graphical modeling as the multivariate analysis method. In the factor analysis system according to claim 8 ,
The factor analysis unit divides a plurality of factors included in the analysis target data into a plurality of groups, performs the graphical modeling for each of one or more factors belonging to the plurality of groups, and integrates the results. A factor analysis system characterized by: the computer
Perform input processing for non-linear analysis target data related to factor analysis,
performing predetermined processing on the data to be analyzed;
extracting data of a linear part from the data to be analyzed that has undergone the input process or from the data to be analyzed that has undergone the predetermined processing;
Analyze the relationship of factors with respect to the prediction target for the extracted data of the linear part, and perform control to quantitatively display the analysis results,
Analyzing the relationship of the factors includes performing variable selection based on analysis results obtained by applying a multivariate analysis method to the extracted linear portion.
A factor analysis method characterized by: Perform input processing for non-linear analysis target data related to factor analysis,
performing predetermined processing on the data to be analyzed;
extracting data of a linear part from the data to be analyzed that has undergone the input process or from the data to be analyzed that has undergone the predetermined processing;
Analyzing the relationship of factors with respect to the prediction target for the extracted data of the linear portion, and performing control to quantitatively display the analysis results , wherein analyzing the relationship of the factors is the extraction performing variable selection based on analysis results obtained by applying a multivariate analysis technique to the linear part obtained ;
A program that causes a computer to execute a process.

Images