CN110580543A - A Power Load Forecasting Method and System Based on Deep Belief Network - Google Patents

Abstract

The invention discloses a power load forecasting method based on a deep belief network, which uses a sparse self-encoding neural network to aggregate historical data of power loads; based on a restricted Boltzmann machine, constructs a composite optimized deep belief network forecasting model The prediction model of the deep belief network, which includes successively from input to output: Gauss-Bernoulli restricted Boltzmann machine, Bernoulli-Bernoulli restricted Boltzmann machine and linear regression output layer, It first uses the unsupervised training method for pre-training, and then uses the BP algorithm with the impulse item to fine-tune the parameters; the aggregated historical data is input into the deep belief network prediction model for prediction. The invention also discloses a power load forecasting system based on a deep belief network. The invention can better excavate the regularity of historical load data, thereby improving the efficiency of prediction, and can fully consider the influence of different factors at the same time, thereby improving the prediction accuracy.

Description

A Power Load Forecasting Method and System Based on Deep Belief Network

technical field

The present invention relates to a power load forecasting method and system, in particular to a power load forecasting method and system based on a deep belief network.

Background technique

At present, short-term load forecasting is the most important for the management and scheduling of the power system. It can provide data support for the power generation plan to determine the power generation plan that best meets the economic requirements, safety requirements, environmental natural requirements and equipment restrictions. , to ensure the economical and safe operation of the power system. At present, with the continuous development and improvement of the power system, higher requirements are put forward for the short-term load forecasting of the power grid. Although there are relatively mature traditional methods, the traditional methods generally have the problem of low prediction accuracy. Although the predicted results have certain reference value, they are still far from the expected level of power companies.

With the continuous progress of modern scientific theoretical research, a new batch of emerging power load forecasting methods have emerged, such as neural network theory, fuzzy mathematics, support vector machines, etc. These are the progress and development of power load forecasting. have been further improved. However, in the existing technology, a single intelligent prediction method is difficult to deal with the challenges brought by multi-dimensional data to the prediction accuracy and efficiency, while the prediction method based on deep belief network can achieve better prediction by combining multiple data processing methods and intelligent algorithms. precision and efficiency.

Contents of the invention

The present invention provides a power load forecasting method and system based on a deep belief network in order to solve the technical problems existing in the known technology.

The technical solution adopted by the present invention to solve the technical problems existing in the known technology is: a power load forecasting method based on a deep belief network, which uses a sparse autoencoder neural network to aggregate historical data of power loads; The Boltzmann machine constructs a compound optimized deep belief network prediction model; the deep belief network prediction model includes in turn from input to output: Gauss-Bernoulli restricted Boltzmann machine, Bernoulli-Bernoulli Using the restricted Boltzmann machine and linear regression output layer, it first uses the unsupervised training method for pre-training, and then uses the BP algorithm with the impulse item to fine-tune the parameters; input the aggregated historical data into the deep belief network forecast in the forecasting model.

Furthermore, when the historical data of electric load is collected, factors such as date, weather, and demand-side management information are comprehensively considered, and each data type is divided to form an input feature vector.

Further, before using the sparse self-encoder neural network to aggregate the data, the historical data of the electric load is normalized and preprocessed.

Further, the sparse self-encoder neural network adopts a two-layer or three-layer neural network; wherein, the two-layer neural network includes an input layer and an output layer, and the three-layer neural network includes an input layer, a hidden layer, and an output layer.

Further, both the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine are constructed with two layers, the two layers include a visible layer and a hidden layer, and two The units between layers are connected to each other, and there is no connection between two units of the same layer.

The present invention also provides a power load forecasting system based on a deep belief network, including a data preprocessing unit and a deep belief network forecasting model unit; wherein: the data preprocessing unit includes a sparse self-encoder neural network subunit, the Sparse self-encoded neural network subunits input historical data of power loads and perform aggregation processing; the deep belief network prediction model unit includes in sequence from input to output: Gauss-Bernoulli restricted Boltzmann machine, Bernoulli Leigh-Bernoulli Restricted Boltzmann Machine and Linear Regression Output Layer, which first use the unsupervised training mode for pre-training, and then use the BP algorithm with impulse items to fine-tune the parameters, which are input to the data pre-processing unit for processing After the historical data, it outputs the predicted value of electric load.

Further, the data preprocessing unit also includes a normalized preprocessing subunit; the normalized preprocessing subunit performs normalized preprocessing on the historical data of electric load, and the processed data is output to the Sparse self-encoding neural network subunits.

Further, the sparse self-encoder neural network subunit includes a two-layer or three-layer neural network; wherein, the two-layer neural network includes an input layer and an output layer, and the three-layer neural network includes an input layer, a hidden layer, and an output layer.

Further, the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine both include two layers, namely a hidden layer and a visible layer, and the layer between the two layers The units in the same layer are connected to each other, and there is no connection between two units in the same layer.

Further, when the deep belief network prediction model unit is being trained, the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine are respectively stacked by one Layer temporary output layer, and then use the unsupervised training mode for pre-training, and then use the BP algorithm with the impulse item to fine-tune the parameters.

The advantages and positive effects of the present invention are: the present invention fully excavates the regularity in the historical load data, then inputs the data feature vector into the sparse self-encoder neural network for feature fusion, uses the DBN model for load prediction, and performs no Supervised training pre-trains the model, and finally fine-tunes the final prediction result through the BP algorithm. The invention can better excavate the regularity of historical load data, thereby improving the efficiency of prediction, and can fully consider the influence of different factors at the same time, thereby improving the prediction accuracy.

The present invention proposes a power load forecasting method and system based on a deep belief network, which improves learning efficiency while improving the existing neural network algorithm learning problem of using historical data. The simulation results show that compared with the traditional neural network algorithm, the prediction accuracy of the present invention is improved.

Comparing with several existing forecasting methods and systems, the average value of forecasting error of electric load forecasting in the four seasons of the present invention is 3.59% MAPE, which is smaller than the other three methods. Considering the influence of temperature, light intensity and usage time electricity price, the electric load forecasting method and system based on deep belief network of the present invention can more fully use the complex relationship between various influencing factors and electric load.

Description of drawings

Fig. 1 is a kind of flow chart of electric load forecasting method based on deep belief network of the present invention;

Fig. 2 is a schematic diagram illustrating the sparse self-encoder neural network structure in the present invention;

Fig. 3 is the Gibbs sampling method schematic diagram among the present invention;

Fig. 4 is the flowchart of deep belief network prediction model of the present invention;

Fig. 5 is a kind of deep belief network forecasting model of the present invention and adopts the comparative test result graph of the DBN model of two BB-RBMs;

Fig. 6 is the graph that a kind of power load forecasting method based on depth belief network of the present invention and BP neural network forecasting method, SVM forecasting method and traditional DBN method compare;

Fig. 7 is a histogram comparison chart of prediction results of a power load prediction method based on a deep belief network of the present invention, a BP neural network prediction method, an SVM prediction method, and a traditional DBN prediction method under the grid-connected condition of renewable energy.

Detailed ways

In order to further understand the invention content, characteristics and effects of the present invention, the following embodiments are enumerated hereby, and detailed descriptions are as follows in conjunction with the accompanying drawings:

The Chinese meanings of English abbreviations in this application are as follows:

DBN: Deep Belief Network prediction model;

RBM: Restricted Boltzmann Machine;

GB-RBM: Gauss-Bernoulli Restricted Boltzmann Machine;

BB-RBM: Bernoulli-Bernoulli Restricted Boltzmann Machine;

AE: self-encoding neural network;

SAE: Sparse Autoencoder Neural Network;

SVM: support vector machine;

BP: backpropagation algorithm;

S-BP: standard error backpropagation algorithm based on gradient descent;

I-BP: An error backpropagation algorithm that adds an impulse item to the weight update rule;

Gibbs: Gibbs sampling;

MAPE: Mean Absolute Percentage Error.

Please refer to Figures 1 to 7, a power load forecasting method based on a deep belief network, which uses a sparse autoencoder neural network to aggregate historical data of power loads; based on a restricted Boltzmann machine, constructs a composite optimization depth Belief network prediction model; the deep belief network prediction model, which includes in turn from input to output: Gauss-Bernoulli restricted Boltzmann machine, Bernoulli-Bernoulli restricted Boltzmann machine and linear The regression output layer, i.e., the output of the Gauss-Bernoulli Restricted Boltzmann Machine as the input of the Bernoulli-Bernoulli Restricted Boltzmann Machine, Bernoulli-Bernoulli Restricted Boltzmann The output of the Mann machine is used as the input of the linear regression output layer, and the input data sequentially passes through the Gauss-Bernoulli restricted Boltzmann machine and the Bernoulli-Bernoulli restricted Boltzmann machine, and the linear regression output layer output. It first uses the unsupervised training method for pre-training, and then uses the BP algorithm with the impulse item to fine-tune the parameters, that is, first uses the unsupervised training method to pre-train the prediction model of the deep belief network, and then uses the BP algorithm with the impulse item Fine-tuning the parameters of the deep belief network prediction model; finally, inputting the aggregated historical data into the deep belief network prediction model for prediction, and obtaining the predicted value of the power system load.

When using the unsupervised training method for pre-training, Gibbs sampling is performed on the training data, and the contrastive divergence CD-k algorithm is used to accelerate the training process of the deep belief network prediction model. During training, Gauss-Bernoulli Restricted Boltzmann Machine and Bernoulli-Bernoulli Restricted Boltzmann Machine can be pre-trained using unsupervised training methods. During training, the training data can be respectively Gibbs sampling processing, and using the contrastive divergence CD-k algorithm to accelerate the training process of Gauss-Bernoulli Restricted Boltzmann Machine or Bernoulli-Bernoulli Restricted Boltzmann Machine.

Factors such as date, weather, and demand-side management information can be considered comprehensively when collecting historical data of electric loads. Weather factors can include temperature, precipitation, wind speed, and solar radiation; day-type factors can include holidays, working days, and demand-side management information factors It can include electricity data and time-of-use electricity price, etc. Each data type is divided in detail to form an input feature vector.

Before using the sparse self-encoder neural network to aggregate the data, the historical data of the electric load can be normalized and preprocessed.

The sparse self-encoder neural network can be a two-layer or three-layer neural network; wherein, the two-layer neural network includes an input layer and an output layer, and the three-layer neural network includes an input layer, a hidden layer, and an output layer.

Both the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine can be constructed using two layers, the two layers can include a visible layer and a hidden layer, and the two layers The units in between can be connected to each other, and there is no connection between any two units on the same layer.

The present invention also provides an embodiment of a power load forecasting system based on a deep belief network, including a data preprocessing unit and a deep belief network forecasting model unit; wherein: the data preprocessing unit includes a sparse self-encoding neural network subunit , the sparse self-encoder neural network subunit inputs the historical data of electric load and performs aggregation processing; the deep belief network prediction model unit, which sequentially includes from input to output: Gauss-Bernoulli restricted Boltzmann machine , a Bernoulli-Bernoulli Restricted Boltzmann Machine and a linear regression output layer, i.e., the output of the Gauss-Bernoulli Restricted Boltzmann Machine as a Bernoulli-Bernoulli Restricted Boltzmann The input of the Mann machine, the output of the Bernoulli-Bernoulli restricted Boltzmann machine is used as the input of the linear regression output layer, and the input data is sequentially passed through the Gauss-Bernoulli restricted Boltzmann machine and Bernoulli - Bernoulli Restricted Boltzmann Machine, output by the linear regression output layer. It first uses the unsupervised training mode for pre-training, and then uses the BP algorithm with impulse items for parameter fine-tuning, which inputs the historical data processed by the data pre-processing unit, and outputs the predicted value of the electric load. That is, the unsupervised training method is used to pre-train the deep belief network prediction model, and then the BP algorithm with the impulse item is used to fine-tune the parameters of the deep belief network prediction model; finally, the aggregated historical data is input into the Prediction is carried out in the deep belief network prediction model, and the predicted value of the power system load is obtained.

Further, the data preprocessing unit may also include a normalized preprocessing subunit; the normalized preprocessing subunit performs normalized preprocessing on the historical data of the electric load, and the processed data is output to the The sparse self-encoder neural network subunit is described.

Further, the sparse self-encoder neural network subunit may include a two-layer or three-layer neural network; wherein, the two-layer neural network includes an input layer and an output layer, and the three-layer neural network includes an input layer, a hidden layer, and an output layer.

Further, the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine may both include two layers, which may be a hidden layer and a visible layer respectively, and the two The units between layers can be connected to each other, and there is no connection between any two units of the same layer.

Further, when the deep belief network prediction model unit is being trained, the Gauss-Bernoulli Restricted Boltzmann Machine and the Bernoulli-Bernoulli Restricted Boltzmann Machine can be stacked separately One layer of temporary output layer, and then each can use the unsupervised training mode for pre-training, and then use the BP algorithm with the impulse item to fine-tune the parameters.

The working principle of the present invention is illustrated below in conjunction with a preferred embodiment of the present invention:

1. Select load data and use SAE to aggregate data

In order to make the prediction results more accurate, a variety of factors affecting the load are considered, including demand-side management information such as date attributes, weather data, peak and valley time-of-use electricity prices, and each data type is divided in detail to form the power load forecasting model. Input feature vector. Then, the feature vectors are put into the autoencoder for feature aggregation to achieve efficient aggregation for nonlinear data.

S1. Selection of load data

The data includes four main groups of measurement variables: weather data (temperature, precipitation, wind speed and solar radiation), day type data, electricity data and time-of-use electricity price.

① Data selection

Non-holidays: The historical data of non-holidays for several days before the forecast date can be selected as the training sample set. Holidays: The commonly used gray relational projection method (Algorithm 1) can be used to select data on similar days to the day to be predicted.

Calculate the correlation between Y _0j and Y _ij :

Where λ is the resolution coefficient.

Calculate the weight of each influencing factor:

② Data preprocessing

The goal of data normalization is to change dimensioned data into dimensionless data and map the data to the range of 0 to 1 (Fig.5), which can improve the convergence speed of the prediction model.

Among them, _Xi is the sample data, is the normalized value of Xi, X _max and X _{min are} the maximum and minimum values of _Xi respectively;

S2. Use SAE to aggregate data

Autoencoder neural network (AE) is a three-layer unsupervised neural network that can take input vectors to form a high-level concept in the next layer through nonlinear mapping. AE tries to approximate the same function, thus bringing the target output value close to the input value thus minimizing the expected reconstruction error. Figure 2 is the basic architecture of AE. The first layer is the input layer, the middle layer is the hidden layer, and the last layer is the output layer. The AE network can perform non-linear transformation from the previous layer to the next layer through the activation function.

The learning process of the AE network consists of two stages: encoder and decoder stages.

The encoder transforms the input into a more abstract feature vector, and the decoder reconstructs the input from the feature vector. The encoder is a forward pass process. The encoder needs to be trained {x ⁽¹⁾ ,x ⁽²⁾ ,...,x ⁽ⁱ⁾ }, x ⁽ⁱ⁾ ∈ Z ^{n is} nonlinearly mapped to the hidden layer through the following sigmoid function f(z):

The decoding process is to reconstruct the input layer in the output. The reconstructed vector {y ⁽¹⁾ ,y ⁽²⁾ ,...,y ⁽ⁱ⁾ } can be given by:

where W is the weight vector between these different layers and b is the offset. {W,b} are trainable parameters in the encoder and decoder.

In a network, a node is considered "inactive" if its output is zero or close to zero in the hidden layer, and "active" when the output is 1 or close to 1. To make a hidden layer sparse, we need to make most of the nodes inactive in this hidden layer. Therefore, a sparsity constraint is imposed on the hidden nodes with the following formula:

in is the average activation value of hidden nodes, the variable m is the number of training times, and [a _j (x ⁽ⁱ⁾ )] represents the activation value of hidden nodes in layer j and samples in layer i.

For the training set, in order to avoid learning the same function and improve the ability to capture important information, it is necessary to set the average activation of each hidden node j to zero or close to zero. Therefore, the additional penalty factor is:

where ρ is the sparsity parameter, and KL( ) is the Kullback-Leibler divergence, which is used as a penalty measure for the desired and actual distribution. if but when As it approaches 0 or 1, the KL-divergence approaches infinity. The SAE cost function can be determined by the following formula:

where J(W,b) is a loss function that aims to make the encoder that produces the output as equivalent to the input as possible. The parameters (W, b) can be updated using a gradient descent algorithm.

2. Constructing a deep belief network model. The deep belief network model of the present invention is an improved deep belief network model of compound optimization, hereinafter referred to as the improved DBN.

Introduction to RBM: Restricted Boltzmann Machine (RBM) is a random neural network (that is, when the neuron nodes of the network are activated, there will be random behavior and random values). It consists of a visible layer and a hidden layer. Neurons in the same layer are independent of each other, while neurons in different network layers are connected to each other (bidirectional connection). When the network is trained and used, information will flow in two directions, and the weights in both directions are the same. But the offset values are different (the number of offset values is the same as the number of neurons), the upper layer of neurons constitutes a hidden layer (hidden layer), and the value of the hidden layer neurons is used as h vector. The neurons in the lower layer form the visible layer, and the v vector represents the value of the neurons in the visible layer. The connection weights can be represented by a matrix W. The difference with DNN is that RBM does not distinguish between forward and reverse, the state of the visible layer can act on the hidden layer, and the state of the hidden layer can also act on the visible layer. The bias coefficient of the hidden layer is the vector b, while the bias coefficient of the visible layer is the vector a.

RBM model parameters: mainly weight matrix w, bias coefficient vectors a and b, hidden layer neuron state vector h and visible layer neuron state vector v.

In this method, the DBN is composed of two RBMs: BB-RBM and GB-RBM:.

S3. Construct BB-RBM model

For Bernoulli RBM (Bernoulli-Bernoulli RBM, BB-RBM), both visible and hidden units are binary random units. Its energy function is:

E(v,h)＝-a ^T vb ^T hv ^T wh (Formula 9)

Among them, weight matrix W, bias coefficient vectors a and b, hidden layer neuron state vector h and visible layer neuron state vector v, w, a, b, h and v are the parameters of RBM. From Fig.5 we can see that the model is divided into two groups of units: v and h, their biases correspond to a and b, and the interaction between them is described by w.

The joint probability distribution of the model is defined in terms of the energy function as:

In the formula, Z=∑v _,he - ^E(v,h) is the partition function.

"Restriction" means that there is no connection between similar nodes of the RBM model, which means that the conditional independence between hidden layer units (or visible units) is established. In BB-RBM, all units are binary random units, which means that the input data should be binary, or a real value between 0 and 1 representing the probability that a visible unit is active or inactive.

The conditional probability distribution for each unit is given by the sigmoid function of the inputs it receives:

In the formula, σ(x)=1/(1+exp(-x)) is the sigmoid activation function.

S4, build GB-RBM model

Different from BB-RBM, the energy function of GB-RBM is defined as:

where σ is the standard deviation of the Gaussian noise of v.

The conditional probability distribution of the model is the same as that of BB-RBM, and the joint probability distribution:

In the formula, v _i takes a real value and obeys a Gaussian distribution with mean μ and variance σ.

S5, Gibbs sampling

Based on the above model, it is necessary to train the RBM, that is, to adjust the parameter θ to fit the given training samples. Maximum likelihood estimation is a commonly used method for estimating model parameters. When applied in this paper, it is to find the parameter θ to maximize the probability of all training data x under this distribution, so the RBM training problem can be transformed into solving the likelihood function most value problem.

Given a training set, the model log-likelihood for each training sample can be expressed as:

where θ = {b, a, W}, D is the training dataset.

Its gradient can be expressed as:

where E _P* is the expected value of x under the empirical distribution P ^* , and E _P is the expected value under the model distribution P.

Although E _P* [x h] can be easily calculated according to the training data; but E _P [x h] corresponds to all possible values of v and h, and it is necessary to traverse all possible values of visible units and hidden units The numerical combination of , and the number of combinations is exponential, so it is difficult to calculate directly. Unbiased samples of E _P can usually be obtained by Gibbs sampling (as shown in Figure 3) for estimating expectations.

S6, CD-k algorithm

In order to speed up the training process of RBM, the contrastive divergence CD-k algorithm is used for unsupervised learning. Because in the CD-k algorithm (k represents the number of sampling times), when k=1, that is, only one step of Gibbs sampling is performed, and a good fitting effect can be achieved. Therefore, the CD-1 algorithm is generally used to fit the values of each parameter.

W←W+ε×[p(h＝1|v)v ^T -p(h ^* ＝1|v ^* )v ^*T ]

b←b+ε×(vv ^* )

c←c+ε×[p(h＝1|v)-p(h ^* ＝1|v ^* )] (Formula 17)

In the formula, the reconstruction of the visible layer v is v ^* , and the hidden layer obtained according to the reconstructed visible layer v ^* is h ^* . Assuming the learning efficiency is ε, after the RBM is trained by the contrastive divergence algorithm, the weight matrix W, the bias vector b of the visible layer, and the bias vector c of the hidden layer.

S7, build an improved DBN model

When DBN is applied to load forecasting under complex factors, the key issues are how to construct a suitable forecasting model and how to effectively train the constructed forecasting model. In the embodiment of the present invention, the DBN model shown in FIG. 4 is improved to be suitable for solving the problem of short-term power load forecasting.

The improved DBN model can be composed of a GB-RBM, multiple BB-RBMs and a linear regression output layer: (1) The GB-RBM is used as the first RBM of the DBN to be stacked, so that the weather data and load in the input data Data and other continuous real-valued data can be effectively converted into binary data; (2) Because BB-RBM is suitable for the modeling process of binary data (such as black and white images or encoded text), other RBMs use BB-RBM, Realize the feature extraction of input data; (3) The hidden layer and output layer of the last RBM form a linear regression network structure, and the feature vector extracted by the improved DBN is used as input, and the time interval obtained by processing the linear activation function can be 15 minutes. 30-minute or 1-hour electrical load time series y.

3. Use unsupervised training to pre-train the model and use BP algorithm to fine-tune the parameters

Unsupervised learning is used to pre-train DBN, which effectively completes the classification task of speech recognition, and provides a better parameter basis for the next step of parameter fine-tuning. In the process of hybrid pre-training, a temporary output layer needs to be stacked on top of the BB-RBM and GB-RBM to be trained to ensure the integrity of the prediction model.

S8, unsupervised pre-training

Applying Unsupervised Learning to Deep Belief Networks. The sparse autoencoder model is used as a preprocessing tool for data in deep learning. The sparse autoencoder model trains the sparse autoencoder parameters. In the sparse autoencoder model, the reconstructed data is searched. make it close to the original data a, that is:

Extract M training samples and calculate the reconstruction error function C:

Among them, y is the coefficient vector, w is the weight coefficient of the training sample. Through the optimization formula:

To get the coefficient y and the weight coefficient w of the sample.

in, is the sparsity penalty factor.

S9. Use BP algorithm to fine-tune parameters

After the layer-by-layer hybrid pre-training provides good network parameters for the improved DBN model, this paper adopts the BP algorithm for global parameter fine-tuning. BP neural network is a multi-layer feed-forward bionic algorithm, which has good characteristics of forward transmission of information and back propagation of errors. By repeating the cycle continuously to achieve the desired error, the desired model is finally obtained after training. However, due to the problems of slow learning speed and low precision in the BP neural network, measures to improve the convergence speed of the backpropagation algorithm are adopted, that is, additional impulse items, namely:

Δω _ij (n+1)=ηδ _j ο _i +αΔω _ij (n) (0<α<1) (Formula 21)

In the formula:

4. Experimental settings

The load test experimental sample data is based on the actual load data of a certain area in China from January 2016 to December 2017. The data includes four main measurement variables: weather data (temperature, precipitation, wind speed and solar radiation), day type Data, electricity data and time-of-use electricity price data (peak hours are 7:00-11:00, 19:00-23:00; normal hours are 11:00-19:00; low-peak hours are 23:00-7 :00). The weather data comes from the meteorological website, and the collection frequency is 1h. The load data is analyzed by hourly data corresponding to the weather data.

In order to compare the performance of the algorithm, among the most commonly used methods for parameter fine-tuning of DBN, the standard BP algorithm based on gradient descent (S-BP for short) and the BP algorithm (I-BP for short) with an impulse item added to the weight update rule are selected. comparing. On the basis of pre-training, two BP algorithms, S-BP and I-BP, are set up for fine-tuning, forming two different optimization strategies "mixed training + I-BP fine-tuning" and "mixed training + S-BP fine-tuning".

The prediction result of the optimization strategy of "mixed training + S-BP fine-tuning" fluctuates the most, which may be due to the fact that the S-BP algorithm tends to cause the loss function to converge to a local optimum for the optimization of multi-layer network parameters; "mixed training + I -BP fine-tuning" is relatively stable. This may be due to the fact that the I-BP algorithm adds an impulse item to a certain extent to increase the search step size, thereby crossing some narrow local minima and reaching smaller places. .

The evaluation and comparison of the models are measured by the mean absolute percentage error (MAPE). Due to the good stability of the MAPE, it can be used as a benchmark for various evaluation standards. The calculation is as follows:

where N represents the number of samples to measure the load, y _l (k) and are respectively expressed as the measured load and predicted load of the first hour of the k-th day.

5. Performance evaluation

(1) The influence of network structure on the prediction model

This paper builds an improved DBN model for short-term power load forecasting, which is the trust in the ability of GB-RBM to process real-valued data. Fig. 5 shows the comparative test results of the improved DBN model and the DBN (B-DBN) model of the model's first RBM adopting BB-RBM. Both models are pre-trained and BP fine-tuned for model parameter optimization.

Fig. 5 is a kind of deep belief network predictive model of the present invention and adopts the comparative test result curve of the DBN model of two BB-RBMs; A kind of deep belief network predictive model of the present invention is a kind of improved DBN, in the figure G-DBN is used to represent a deep belief network prediction model of the present invention; B-DBN is used to represent the DBN model using two BB-RBMs; Figure 5 is a comparison of the prediction results of the two models of G-DBN and B-DBN. The instability of B-DBN's prediction performance is due to the fact that BB-RBM is prone to noise when dealing with real-valued data. On the contrary, the prediction performance of G-DBN is better.

(2) Comparison of different forecasting methods

In order to further verify the feasibility of this embodiment, the loads of a certain region in the four seasons of 2017 are forecasted separately. The training sample set and the test sample set are composed of historical data of 10 months before the forecast date. Choose commonly used artificial intelligence prediction methods: BP neural network, SVM method and traditional DBN method (unsupervised learning pre-training and S-BP algorithm fine-tuning) for comparison. In order to ensure objectivity, the experimental results are the average value obtained by performing 100 experiments.

Please refer to Fig. 6, in Fig. 6, represent a kind of deep belief network prediction model of the present invention with G-DBN, represent traditional DBN model with DBN; The prediction of different methods is compared, a kind of deep belief network prediction of the present invention The prediction error of the model: the average value of the four seasons is 3.59% MAPE, which is smaller than the other three methods. Considering the influence of temperature, light intensity and usage time electricity price, the improved DBN can more fully adopt the complex relationship between various influencing factors and electric load.

With the increasing penetration of renewable energy power generation (mainly photovoltaic power generation and wind power), more than 20% of the annual electricity demand in some countries comes from wind power and solar power, and the output of photovoltaic power generation and wind power in some hours in some areas even exceeds 50% of the load, the problem of fluctuation and uncertainty in the operation of the power system is more prominent. In order to verify the generalization performance of the method of the present invention, the loads of three different regions where the proportion of renewable energy power generation output is about 30%, about 20%, and about 10% are obtained as input samples for the comparison test, and MAPE is used as the evaluation index .

Please refer to FIG. 7, in which G-DBN is used to represent a deep belief network prediction model of the present invention, and DBN is used to represent the traditional DBN model. Figure 7 shows that when the proportion of renewable energy output increases, the prediction errors of each method will increase, and the changes of BP and SVM are more obvious, while the traditional DBN model and a deep belief network prediction model of the present invention have slight changes. This may be because as the proportion of renewable energy output increases, the operation of the power system becomes more unstable, the nonlinear load curve becomes more complex, and the advantages of deep network fitting complex nonlinear curves are more obvious.

Power load forecasting is an important part of power system planning and the basis of power system economic operation, which provides an important basis for distribution network management decision-making and operation mode. A power load forecasting method based on a deep belief network proposed by the present invention improves the learning efficiency of the existing neural network algorithm learning and adopting historical data. The simulation results show that compared with the traditional neural network algorithm, the prediction accuracy of the electric load forecasting method based on the deep belief network is improved.

The present invention also provides a preferred embodiment of a power load forecasting system based on a deep belief network, which includes a data preprocessing unit and a deep belief network prediction model unit; wherein: the data preprocessing unit includes normalized preprocessing A subunit and a sparse self-encoding neural network subunit, the normalized preprocessing subunit performs normalized preprocessing on the historical data of the electric load, and the processed data is input to the sparse self-encoding neural network subunit, The sparse self-encoder neural network subunit aggregates the historical data of the input power load; the deep belief network prediction model unit includes in sequence from input to output: Gauss-Bernoulli restricted Boltzmann machine , Bernoulli-Bernoulli Restricted Boltzmann Machine and Linear Regression Output Layer, it first uses the unsupervised training mode for pre-training, and then uses the BP algorithm with the impulse item to fine-tune the parameters, and it inputs the data pre-training The historical data processed by the processing unit outputs the predicted value of electric load.

The data preprocessing unit comprehensively considers various factors that affect the load such as date, weather, and demand-side management information, and divides each data type in detail to form the input feature vector of the power load forecasting model; input the feature vector feature To multiple two-layer sparse autoencoder neural networks for feature fusion.

Deep belief network prediction model unit; unsupervised training is used to pre-train the model, and BP algorithm is used to fine-tune parameters; the output data is obtained from the load data training, and the predicted value of the power system load is obtained.

Deep belief network prediction model unit, including: GB-RAM, BB-RAM and linear regression output layer, GB-RAM is composed of hidden layer and visible layer, the units of the two layers are connected to each other, but the same layer units are not connected Link exists. BB-RAM consists of a hidden layer and a visible layer. The units in the two layers are connected to each other, but there is no link between the units in the same layer.

The energy function of GB-RBM is

Among them, w, a and b are the parameters of RBM. v and h, their biases correspond to a and b, and the interaction between them is described by w, where σ is the standard deviation of the Gaussian noise of v.

The energy function of BB-RBM is

E(v,h)＝-a ^T vb ^T hv ^T wh

The deep belief network prediction model unit is specifically used to train the model using the unsupervised training method to optimize the parameters of the deep belief network prediction model unit. During the optimization process, a layer is stacked on top of GB-RAM and BB-RAM Temporary output layer to ensure the integrity of the prediction model; the I-BP algorithm is used to globally fine-tune the prediction model units of the deep belief network to determine the topology of the prediction model units of the deep belief network.

The output data of the deep belief network prediction model unit becomes the predicted value of the power system load.

The invention uses an autoencoder to aggregate comprehensive historical data, uses a multi-layer restricted Boltzmann machine to form a deep belief network prediction model unit, and trains the model without supervision, thereby improving learning performance and prediction accuracy. While improving the existing neural network algorithm to learn and analyze historical data, it also improves the learning efficiency. The simulation results show that, compared with the traditional neural network forecasting system, the power load forecasting system based on the deep belief network of the present invention has improved power load forecasting accuracy.

The above-described embodiments are only used to illustrate the technical ideas and characteristics of the present invention, and its purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The present invention cannot be limited only by this embodiment. The scope of the patent, that is, all equivalent changes or modifications made to the spirit disclosed in the present invention still fall within the scope of the patent of the present invention.

Claims (10)

1. A power load prediction method based on a deep belief network is characterized in that a sparse self-coding neural network is adopted to carry out aggregation processing on historical data of a power load; constructing a composite optimized deep belief network prediction model based on a restricted Boltzmann machine; the deep belief network prediction model comprises the following components in sequence from input to output: the system comprises a Gaussian-Bernoulli limited Boltzmann machine, a Bernoulli-Bernoulli limited Boltzmann machine and a linear regression output layer, wherein the Gaussian-Bernoulli limited Boltzmann machine and the Bernoulli-Bernoulli limited Boltzmann machine are pre-trained by adopting an unsupervised training method, and then parameter fine tuning is carried out by adopting a BP algorithm with an impulse term; and inputting the aggregated historical data into the deep belief network prediction model for prediction.

2. The deep belief network-based power load prediction method of claim 1, wherein historical data of the power load is collected by dividing each data type to form an input feature vector, taking into account factors of date, weather and demand side management information.

3. The deep belief network-based power load prediction method of claim 1, wherein a normalization pre-processing is performed on historical data of the power load before the data are aggregated by using a sparse self-coding neural network.

4. The deep belief network-based power load prediction method of claim 1, wherein the sparse self-encoding neural network employs a two-layer or three-layer neural network; the two layers of neural networks comprise an input layer and an output layer, and the three layers of neural networks comprise an input layer, a hidden layer and an output layer.

5. The deep belief network-based power load prediction method of claim 1, wherein the gaussian-bernoulli limited boltzmann machine and the bernoulli-bernoulli limited boltzmann machine are each constructed using two layers, the two layers include a visible layer and a hidden layer, the units between the two layers are connected with each other, and no connection exists between every two units on the same layer.

6. a power load prediction system based on a deep belief network is characterized by comprising a data preprocessing unit and a deep belief network prediction model unit; wherein: the data preprocessing unit comprises a sparse self-coding neural network subunit, and the sparse self-coding neural network subunit inputs historical data of the power load and performs aggregation processing; the deep belief network prediction model unit sequentially comprises the following components from input to output: the system comprises a Gaussian-Bernoulli limited Boltzmann machine, a Bernoulli-Bernoulli limited Boltzmann machine and a linear regression output layer, wherein the Gaussian-Bernoulli limited Boltzmann machine, the Bernoulli-Bernoulli limited Boltzmann machine and the linear regression output layer are pre-trained by adopting an unsupervised training mode, then parameter fine tuning is carried out by adopting a BP algorithm added with a momentum term, historical data processed by a data preprocessing unit are input, and a predicted value of the power load is output.

7. the deep belief network-based power load prediction system of claim 6, wherein the data pre-processing unit further comprises a normalization pre-processing subunit; and the normalization preprocessing subunit performs normalization preprocessing on the historical data of the power load, and the processed data is output to the sparse self-coding neural network subunit.

8. the deep belief network-based power load prediction system of claim 6, wherein the sparse self-encoding neural network subunit comprises a two-layer or three-layer neural network; the two layers of neural networks comprise an input layer and an output layer, and the three layers of neural networks comprise an input layer, a hidden layer and an output layer.

9. The deep belief network-based power load prediction system of claim 6, wherein the Gaussian-Bernoulli limited Boltzmann machine and the Bernoulli-Bernoulli limited Boltzmann machine each comprise two layers, a hidden layer and a visible layer, respectively, wherein the units between the two layers are connected to each other, and no connection exists between any two units on the same layer.

10. The deep belief network-based power load prediction system of claim 9, wherein when the deep belief network prediction model unit is trained, the gaussian-bernoulli limited boltzmann machine and the bernoulli-bernoulli limited boltzmann machine are each stacked with a temporary output layer, and then are each pre-trained in an unsupervised training mode, and then are further parameter-fine-tuned by using a Back Propagation (BP) algorithm with an impulse term.