CN105305426B - Mapreduce two-step short-period load prediction method based on deviation control mechanism - Google Patents

Abstract

The invention discloses a Mapreduce two-step short-period load prediction method based on a deviation control mechanism. The method comprises the steps that 1) an ARMA method is used to obtain a relative error value of prediction without consideration of influential factors of load; and an SVM method is used to implement secondary load prediction by taking the relative error value of prediction and the daily average temperature of a point to be predicted as the influential factors, the prediction value is corrected on the basis of prediction of the first step, and the deviation of load is controlled and predicted. The ARMA method and the SVM method are used simultaneously to realize Mapreduce, the speed-up ratio and expansibility are higher, the real-time temperature data is utilized, parallel operation prediction is carried out in the Hadoop platform, and short-period real-time load prediction is realized on the basis that the prediction speed and precision are ensured.

Description

Mapreduceization two-step method short-term load forecasting method based on deviation control mechanism

Technical field

The present invention relates to the Methods of electric load forecasting for sending out a kind of based on big data, is based particularly on deviation control mechanism Load forecasting method.

Background technology

The class short-term load forecasting method that time series models are considered as most classical, most system, are most widely adopted, But which has the disadvantage that：1. normally run in electrical network, the prediction when factor change such as weather is little it is more accurate, but in randomness When factor is changed greatly or there is bad data, predict the outcome then not ideal；2. other influences load prediction can not be processed very well Factor；3. it is poor in daily load Feng Gu turning point precision of predictions.

And a good load forecasting method should meet：1. hourly weather factors can be processed；2. according to the deviation predicted not The structure and parameter of disconnected adjustment model, constitutes a closed loop feedback.Therefore the series model that should employ one's time to the best advantage is measured in advance To relative error magnitudes account for the secondary load prediction of hourly weather factors, realize control to load prediction deviation.

The content of the invention

It is an object of the invention to provide a kind of two-step method short-term load forecasting algorithm for realizing deviation control, to solve tradition Time series forecasting algorithm is poor in daily load Feng Gu turning point precision of predictions, it is impossible to real-time processing meteorological factor, effectively profit Use model prediction deviation, the low problem of caused load prediction precision.

The present invention provides a kind of Mapreduceization two-step method based on deviation control mechanism for solving above-mentioned technical problem Short-term load forecasting method, the first step adopt autoregressive moving-average model Autoregressive Moving-Average Model (ARMA) method, does not consider the influence factor of load, obtains Relative Error；Second step adopts support vector machine Support Vector Machine (SVM) methods, it is considered to the impact of the Relative Error of the first step, in real time to be predicted temperature To carry out secondary load prediction, predictive value is corrected on the fundamentals of forecasting of the first step, realize the real-time estimate to load.It is concrete to walk It is rapid as follows：

Step 1, configuration Linux running environment, build distributed type assemblies Hadoop platform, configure distributed file system HDFS, concurrent operation Mapreduce；

Step 2, data acquisition：Historical load data is gathered from the EMS system of local power supplying companies bureaus to be predicted, from meteorology Historical temperature data is gathered in the data base of office, the total sample number of two kinds of data is M, and daily sample frequency is f, and adopts The sample moment is identical；

Step 3, data prediction：Historical load data, historical temperature data are normalized, by historical load Data are divided into weight training collection：L_r={ l_r, r=1,2 ... Q, and Road test collection：L_u={ l_u, u=1,2 ... S, its Middle l_rWith l_uFor historical load data sample, Q+S=M；Historical temperature data is divided into into temperature training set：T_r={ t_r, r= 1,2 ... Q, and temperature test collection：T_u={ t_u, u=1,2 ... S, wherein t_rWith t_uFor historical temperature data sample, Q+S= M；Historical load data is identical with the division methods of historical temperature data.

Step 4, by autoregressive moving-average model ARMA method Mapreduceization, carry out a load prediction, concrete steps It is as follows：

4A. weight trainings collection is split：It is determined that the number of parallel ARMA is N, the configuration file of HDFS is changed so that load is instructed Practice collection and be divided into N number of historical load data subset, be uploaded to HDFS file system；

4B. designs Map functions：Model parameter is obtained using ARMA methods, and is exported as value values；

4C. designs Reduce functions：The arma modeling parameter that N number of Map tasks are obtained is carried out averagely, obtaining final ARMA Model, is predicted to all of historical load, and calculates relative error E according to the following formula₁, while being stored in HDFS file system：

Wherein：E₁It is the Relative Error of the load to be predicted obtained using ARMA methods, l_F1It is to be carried out using ARMA methods The load prediction results that load prediction is obtained, l_RFor load actual value；

Step 5, data compilation：Combined training collection：x_r=(E_1r,t_r,l_r), r=1,2 ... Q, integration test collection：x_u= (E_1u,t_u,l_u), u=1,2 ... S, wherein l_r、l_uRepresent historical load value in step 3, E_1r、E_1uThe l that representative is obtained by step 4_r Corresponding relative error magnitudes, t_r、t_uRepresent l_r、l_uHistorical temperature value in the step 3 of point correspondence；

Step 6, by SVM method Mapreduceization, carry out secondary load prediction, comprise the following steps that：

6A. combined trainings collection is split：It is determined that the number of parallel SVM is N, the configuration file of HDFS is changed so that comprehensive to instruct Practice collection data and be divided into the sub- training set of N number of synthesis, be uploaded to HDFS file system；

6B. designs Map functions：Select radial direction base as kernel function, select kernel functional parameter and penalty factor, to comprehensive son Training set carries out SVM model trainings, obtains sub- supporting vector, and the value as Map functions is exported；

6C. designs Reduce functions：The sub- supporting vector for collecting N number of Map functions output is always supported vector set, then right It is total to support that vector set carries out SVM training, obtain optimum Lagrange multipliera^*And optimal threshold b^*, it is pre- that foundation carries out load The SVM regression functions of survey：

In formula, x_r=(x_r1,x_r2,y_r), r=1,2 ... Q are training sample, x_uFor the feature value vector of point to be predicted；

Load prediction is carried out using SVM regression functions and obtains the L that predicts the outcome_Fi2, and relative error is calculated according to the following formula：

Wherein：E₂It is the Relative Error of the load to be predicted obtained using SVM methods, L_F2Adopt that SVM methods obtain for Predicted load, l_RFor load actual value；The secondary load that SVM methods are obtained is predicted the outcome HDFS is stored in relative error result File system.

Compared with conventional art, the invention has the beneficial effects as follows：

When conventional time series Forecasting Methodology is applied to load prediction, it is impossible to effectively consider to affect load prediction precision because Element, while which is poor in daily load Feng Gu turning point precision of predictions, and existing other load forecasting methods are not all accounted for Prediction error value is made full use of, the present invention is with the prediction error value of time series forecasting and the predicted temperature work with point to be predicted The input feature vector of SVM methods is predicted for secondary load, Time Series Forecasting Methods is eliminated pre- in daily load Feng Gu turning points The problem of low precision is surveyed, while utilizing real time temperature data, the parallelization operation Forecasting Methodology in Hadoop platform is ensureing pre- Realize on the basis of degree of testing the speed and precision of prediction that short-term Real-time Load is predicted.

Description of the drawings

Fig. 1 is flow process of the present invention based on the Mapreduceization two-step method short-term load forecasting method of deviation control mechanism Figure.

Fig. 2 is the ARMA load prediction flow charts of first step Mapreduceization.

Fig. 3 is the SVM load prediction flow charts of second step Mapreduceization.

Specific embodiment

For becoming apparent the present invention, accompanying drawing is coordinated to be described in detail below,

A kind of Mapreduceization two-step method short-term load forecasting method based on deviation control mechanism, concrete steps include：

Step 1, configuration Linux running environment, build distributed type assemblies Hadoop platform, configure distributed file system HDFS, concurrent operation Mapreduce；

Step 2, data acquisition：EMS system from local power supplying companies bureaus to be predicted, in the data base of weather bureau, gather history Load data, historical temperature data, the total sample number of two kinds of data are M, and daily sample frequency is f, and sampling instant It is identical；

Step 3, data prediction：Historical load data, historical temperature data are normalized, by historical load Data are divided into weight training collection：L_r={ l_r, r=1,2 ... Q, and Road test collection：L_u={ l_u, u=1,2 ... S, its Middle l_rWith l_uFor historical load data sample, Q+S=M；Historical temperature data is divided into into temperature training set：T_r={ t_r, r= 1,2 ... Q, and temperature test collection：T_u={ t_u, u=1,2 ... S, wherein t_rWith t_uFor historical temperature data sample, Q+S= M；Historical load data is identical with the division methods of historical temperature data.

Step 4, load prediction is carried out using autoregressive moving-average model ARMA methods, comprised the following steps that：

4A. weight trainings collection is split：It is determined that the number of parallel ARMA is N, the configuration file of HDFS is changed so that load is instructed Practice collection and be divided into N number of historical load data subset, be uploaded to HDFS file system；

4B. designs Map functions：Model parameter is obtained using traditional ARMA methods, and is exported as value values：

Step1:Historical load data to gathering carries out stationary test, using non parametric testss method：By electric load Data make scatterplot, and the Power system load data at the moment on the same day uses+1 table when being more than this section of period moment average power load Show, the Power system load data at the moment on the same day represents with -1 when being less than this section of period moment average power load that γ represents trip Number of passes (positive and negative staggeredly to count), N represents+1 occurrence number, and M represents -1 occurrence number, then statistic is：N=N+M.Significant level α=0.05 is taken, is gone through when | Z |≤1.96 History load data is stationary random sequence, is otherwise non-stationary random series.

Step2:If electric load sequence is non-stationary series, pretreatment is carried out to the load sequence：Tranquilization process, Pulverised process；

Tranquilization is processed：Difference processing, first-order difference are carried out to load data：ΔX_t=X_t-X_t-1, then according to Step1 Judge the stationarity of the sequence Jing after processing, if not stationary sequence then proceeds difference, until it becomes stationary sequence.

Pulverised process：As load data sequence { x_t, during the average non-zero of t=1,2 ... N, construct zero-mean sequence：y_t= x_t-Ex_t, wherein

Step3:Approach For Identification of Model Structure：

1. calculate steady, zero-mean the random sequences { y obtained by Step2_t, t=1, the auto-correlation function of 2 ... N：

2. calculate partial autocorrelation function：

3. m is given, n initial values, if ρ_kIn certain k>After mThen { y_tFor MA (0, m) sequence； IfIn certain k>After nThen { y_tIt is AR (n, 0) sequence；If ρ_kWithBe not 0, then { y_tIt is ARMA sequences, But not can determine that exponent number.

Step4:Parameter estimation:

If 4. AR (n) models：WhereinIt is constant, a_tFor white noise parameter estimation it is：

Wherein：

If 5. MA (m) models：y_t=-θ₁a_t-1-θ₂a_t-2-…-θ_ma_t-m+a_t, wherein θ₁、…θ_mIt is constant, a_tFor white Noise.General MA (m) model orders are 1 or 2, and exponent number is further added by then more complicated, therefore directly takes m=1 or m=2, using direct Method seeks parameter estimation.

During m=1:

During m=2:

If 6. ARMA (n, m) model：Whereinθ₁、…θ_m It is constant, a_tFor white noise.

1) make p_N=(lnN)^1+δ, 0≤δ≤1 is any given value, by 4. calculating

2) seek long auto-regression model residual error：T=p_N+ 1 ..., N, using residual error auto-correlation function Inspection { a_tIndependence, if not independent, increase p_N, turn 1), otherwise to turn 3)；

3) according to the parameter of least-squares estimation ARMA (n, m) model：β=(X^TX)^-1X^TY

Wherein：

4) model order is determined using AIC criterion：

5) m, n are selected again, to m, n repeat 1)～4), selection makes one group of minimum m of AIC (n, m)^*,n^*As 3) exponent number of ARMA, by obtaining parameter estimation.

4C. designs Reduce functions：The arma modeling parameter that N number of Map tasks are obtained is carried out averagely, obtaining final ARMA Model, is predicted to all of historical load, and calculates relative error E according to the following formula₁, while being stored in HDFS file system：

Wherein：E₁It is the Relative Error of the load to be predicted obtained using ARMA methods, l_F1It is to be carried out using ARMA methods The load prediction results that load prediction is obtained, l_RFor load actual value；

Step 5, data compilation：Combined training collection：x_r=(E_1r,t_r,l_r), r=1,2 ... Q, integration test collection：x_u= (E_1u,t_u,l_u), u=1,2 ... S, wherein l_r、l_uRepresent historical load value, E_1r、E_1uThe l that representative is obtained by step 4_rCorresponding phase To error amount, t_r、t_uRepresent l_r、l_uThe corresponding temperature value of point；

Step 6, by SVM method Mapreduceization, carry out secondary load prediction, comprise the following steps that：

6A. combined trainings collection is split：It is determined that the number of parallel SVM is N, the configuration file of HDFS is changed so that comprehensive to instruct Practice collection data and be divided into the sub- training set of N number of synthesis, be uploaded to HDFS file system；

6B. designs Map functions：Select radial direction base as kernel function, select kernel functional parameter and penalty factor, to comprehensive son Training set carries out SVM model trainings, obtains sub- supporting vector, and the value as Map functions is exported；

6C. designs Reduce functions：The sub- supporting vector for collecting N number of Map functions output is always supported vector set, then right It is total to support that vector set carries out SVM training, obtain optimum Lagrange multiplierα^*And optimal threshold b^*, it is pre- that foundation carries out load The SVM regression functions of survey：

In formula, x_r=(x_r1,x_r2,y_r), r=1,2 ... Q are training sample, x_uFor the feature value vector of point to be predicted；

Load prediction is carried out using SVM regression functions and obtains the L that predicts the outcome_Fi2, and relative error is calculated according to the following formula：

Wherein：E₂It is the Relative Error of the load to be predicted obtained using SVM methods, L_F2Adopt that SVM methods obtain for Predicted load, l_RFor load actual value；The secondary load that SVM methods are obtained is predicted the outcome HDFS is stored in relative error result File system.

Meaning of the present invention shows：(1) the Relative Error value that ARMA methods can be made full use of to obtain so that daily The poor precision of prediction value of load Feng Gu turning points be changed into useful information；(2) consider real-time meteorological factor；(3) realize negative Lotus prediction deviation is controlled.

Claims (1)

1. a kind of Mapreduceization two-step method short-term load forecasting method based on deviation control mechanism, the first step is using from returning Return moving average model ARMA methods, obtain Relative Error；Second step adopts SVM methods, it is considered to which the prediction of the first step is relative by mistake Difference, the impact of daily mean temperature correct predictive value on the fundamentals of forecasting of the first step carrying out secondary load prediction, realize to negative The real-time estimate of lotus, it is characterised in that specifically include following steps：

Step 1, configuration Linux running environment, build distributed type assemblies Hadoop platform, configuration distributed file system HDFS, Concurrent operation Mapreduce；

Step 2, data acquisition：Historical load data is gathered from the EMS system of local power supplying companies bureaus to be predicted, from weather bureau Historical temperature data is gathered in data base, and the total sample number of two kinds of data is M, and daily sample frequency is f, and when sampling Carve identical；

Step 3, data prediction：Historical load data, historical temperature data are normalized, by historical load data It is divided into weight training collection：L_r={ l_r, r=1,2 ... Q, and Road test collection：L_u={ l_u, u=1,2 ... S, wherein l_r With l_uFor historical load value, Q+S=M；Historical temperature data is divided into into temperature training set：T_r={ t_r, r=1,2 ... Q, with And temperature test collection：T_u={ t_u, u=1,2 ... S, wherein t_rWith t_uFor historical temperature data sample, Q+S=M；Historical load Data are identical with the division methods of historical temperature data；

Step 4, by autoregressive moving-average model ARMA method Mapreduceization, carry out a load prediction, concrete steps are such as Under：

4A. weight trainings collection is split：It is determined that the number of parallel ARMA is N, the configuration file of HDFS is changed so that weight training collection N number of historical load data subset is divided into, HDFS file system is uploaded to；

4B. designs Map functions：Model parameter is obtained using ARMA methods, and is exported as value values；

4C. designs Reduce functions：The arma modeling parameter that N number of Map tasks are obtained is carried out averagely, obtaining final ARMA moulds Type, is predicted to all of historical load, and calculates Relative Error E according to the following formula₁, while being stored in HDFS files system System：

$E_1=\left(\frac{l_{F1}-l_R}{l_R}\right)\×100\%$

Wherein：E₁It is the Relative Error of the load to be predicted obtained using ARMA methods, l_F1It is that load is carried out using ARMA methods The load prediction results that prediction is obtained, l_RFor load actual value；

Step 5, data compilation：Combined training collection：x_r=(E_1r,t_r,l_r), r=1,2 ... Q, integration test collection：x_u=(E_1u,t_u, l_u), u=1,2 ... S, wherein l_r、l_uRepresent historical load value in step 3, E_1r、E_1uRepresentative is relative by the prediction that step 4 is obtained Error amount, t_r、t_uRepresent l_r、l_uHistorical temperature value in the step 3 of point correspondence；

Step 6, by SVM method Mapreduceization, carry out secondary load prediction, comprise the following steps that：

6A. combined trainings collection is split：It is determined that the number of parallel SVM is N, the configuration file of HDFS is changed so that combined training collection Data are divided into the sub- training set of N number of synthesis, are uploaded to HDFS file system；

6B. designs Map functions：Select radial direction base as kernel function, select kernel functional parameter and penalty factor, to comprehensive son training Collection carries out SVM model trainings, obtains sub- supporting vector, and the value as Map functions is exported；

6C. designs Reduce functions：The sub- supporting vector for collecting N number of Map functions output is always supported vector set, then to general branch Holding vector set carries out SVM training, obtains optimum Lagrange multiplierα^*And optimal threshold b^*, setting up carries out load prediction SVM regression functions：

$y^{*}\left(x\right)=\underset{r=1}{\overset{M}{\Σ}}({\widehat{\mathrm{\α}}}_i^{*}-{\mathrm{\α}}_i^{*})K(x_r,x_u)+b^{*}$

In formula, x_r=(x_r1,x_r2,y_r), r=1,2 ... Q are training sample, x_uFor the feature value vector of point to be predicted；

Load prediction is carried out using SVM regression functions and obtains the L that predicts the outcome_Fi2, and relative error is calculated according to the following formula：

$E_2=\left(\frac{L_{F2}-l_R}{l_R}\right)\×100\%$

Wherein：E₂It is the Relative Error of the load to be predicted obtained using SVM methods, L_F2Adopt that SVM methods obtain for load Predictive value, l_RFor load actual value；The secondary load that SVM methods are obtained is predicted the outcome HDFS files are stored in relative error result System.