CN112529283A - Comprehensive energy system short-term load prediction method based on attention mechanism - Google Patents
Comprehensive energy system short-term load prediction method based on attention mechanism Download PDFInfo
- Publication number
- CN112529283A CN112529283A CN202011410725.XA CN202011410725A CN112529283A CN 112529283 A CN112529283 A CN 112529283A CN 202011410725 A CN202011410725 A CN 202011410725A CN 112529283 A CN112529283 A CN 112529283A
- Authority
- CN
- China
- Prior art keywords
- short
- model
- lstm
- load
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 49
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000013277 forecasting method Methods 0.000 claims abstract description 6
- 230000015654 memory Effects 0.000 claims description 22
- 238000010606 normalization Methods 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 238000012360 testing method Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 230000002457 bidirectional effect Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 210000002569 neuron Anatomy 0.000 claims description 8
- 210000004027 cell Anatomy 0.000 claims description 6
- 230000006403 short-term memory Effects 0.000 claims description 6
- 230000007774 longterm Effects 0.000 claims description 5
- CGFFKDRVHZIQHL-UHFFFAOYSA-N 1-but-3-en-2-yl-3-(methylcarbamothioylamino)thiourea Chemical compound CNC(=S)NNC(=S)NC(C)C=C CGFFKDRVHZIQHL-UHFFFAOYSA-N 0.000 claims description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 230000001413 cellular effect Effects 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 230000002441 reversible effect Effects 0.000 claims description 3
- 238000001816 cooling Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims description 2
- 230000001737 promoting effect Effects 0.000 claims description 2
- 230000002829 reductive effect Effects 0.000 claims description 2
- 230000007787 long-term memory Effects 0.000 claims 2
- 230000005484 gravity Effects 0.000 claims 1
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 abstract description 12
- 239000003345 natural gas Substances 0.000 abstract description 6
- 238000005457 optimization Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000005611 electricity Effects 0.000 description 3
- 239000007789 gas Substances 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000010248 power generation Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Strategic Management (AREA)
- Artificial Intelligence (AREA)
- Human Resources & Organizations (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Tourism & Hospitality (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a comprehensive energy system short-term load forecasting method based on an attention mechanism, which relates to the field of energy load forecasting and comprises the following steps of 1: preprocessing input data; step 2: establishing a CNN-LSTM-BILSTM model based on an attention system: and step 3: and training the model and predicting the short-term load of the comprehensive energy system according to the training model. According to the technical scheme, the influence of temperature, cold load, electric load data and natural gas load on the electric load of the comprehensive energy system is considered, useful information in historical data can be fully mined, technical support is provided for operation optimization of comprehensive energy, and the method has great practical value.
Description
Technical Field
The invention relates to the field of energy load prediction, in particular to a comprehensive energy system short-term load prediction method based on an attention mechanism.
Background
Energy is a necessity for human life and development. With the development of industry, how to solve the shortage of non-renewable energy sources and environmental pollution has become a challenge of power systems. The use of Integrated Energy Systems (IES) in combination with various energy systems (e.g., cold/heat/electricity/gas systems) allows for the total consumption of renewable energy and the cascading use of non-renewable energy. Therefore, IES is a trend of future development of power systems. However, the complementarity of multiple energy sources also causes great difficulty in power load prediction. Conventional power system load forecasting typically takes into account weather, economics, electricity prices, and holidays. For the IES, the influence of cold (heat) source, natural gas, solar energy, etc. on the system load needs to be considered. Furthermore, due to the time series and non-linearity of the load, and the power system is greatly affected by the demand side, it is difficult to predict the long-term load of the IES. The short-term load forecast refers to load of each hour or each day after load forecast according to historical data of previous days, and provides theoretical basis for the IES to make a power generation plan. The idea of the IES has just emerged, and how to accurately predict the power load of the IES becomes a problem to be solved.
The main prediction methods for conventional power system load prediction include autoregressive differential moving average (ARIMA) and regression analysis. With the rise of big data and artificial intelligence technology of power systems, machine learning and deep learning are widely used for short-term load prediction of power systems, such as support vector machine regression (SVR), Artificial Neural Network (ANN), and the like, and the accuracy of model prediction can be remarkably improved by using the artificial intelligence technology. Among them, SVR can be regarded as convex optimization, and can always find a global optimal solution, which performs well in a smaller data set, but is easily over-fitted when the data set becomes larger. In addition, the slack variables and kernel parameters of the SVR need to be set manually. The ANN has strong data fitting ability, but it is difficult to find a global optimal solution. In most cases, it will fall into a local optimum and conventional artificial neural networks do not work well in processing time-dependent data. A Recurrent Neural Network (RNN) is an improved ANN that retains information about delivery time in the network. RNNs address the problem of conventional ANNs handling time-related data. But during the RNN training process, gradient explosions and gradient disappearing are likely to occur. To address this problem, scholars have proposed Long Short Term Memory (LSTM) networks, which are improvements to RNNs. LSTM can effectively solve RNN problems during training and perform better in longer time series data, and LSTM has been applied in load prediction for power systems. Bi-directional LSTM (BilSTM) is a combination of forward and backward LSTM and can fit data from both the forward and backward directions of the sequence to achieve higher prediction accuracy. Convolutional Neural Networks (CNNs) can extract and create features through convolutional and pooling layers. Also, CNN can be used in short term load prediction to extract or create load features. To better handle the complex influencing factors of IES and the time series and non-linear characteristics of load, we propose a CNN-LSTM-BilSTM load prediction model based on the attention mechanism.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiments of the present invention provide a method for predicting short-term load of an integrated energy system based on attention mechanism. The technical scheme is as follows:
according to a first aspect of the disclosed embodiments of the present invention, there is provided an attention-based method for predicting short-term load of an integrated energy system, comprising:
step 1: preprocessing input data;
step 2: establishing a CNN-LSTM-BILSTM model based on an attention system:
and step 3: and training the model and predicting the short-term load of the comprehensive energy system according to the training model.
In one embodiment, in the input data preprocessing step, it is specifically,
step 1.1: performing characteristic normalization on input data;
step 1.2: serializing the data obtained in step 1.1;
step 1.3: taking 80% of the data set obtained in the step 1.2 as a training set, taking 20% of the data set as a test set, and using the data of the training set to train the model; and the test set is used for testing the trained model, and the error on the test set is used as the generalization error of the final model in the corresponding real scene to distribute the data proportion to the training set and the test set.
In one embodiment, in the step 1.1 of performing feature integration on the input data, a feature normalization formula is used:
wherein a isiIs the ith data to be normalized, aimax,aiminIs the maximum minimum value before normalization, aniIs the normalized data.
In one embodiment, said step 1.2: serializing the data obtained in step 1.1, specifically,
the method comprises generating input non-time sequence data into time sequence features by using a sliding window method, expressing the load and influence factor value at a certain moment by using related time sequence features, expressing the historical load at each moment by using related features together, and realizing the data proportion distribution of a training set and a test set, wherein the sliding window method specifically comprises the following steps of,
setting the width of a sliding window as an hourly record in T days and setting the step length by taking the hourly electric load, the temperature, the cooling load and the air consumption of the current time for T days as input characteristics of a model; and gradually sliding a window, and jointly representing the historical load at each moment by the relevant characteristics of T days.
In one embodiment, the step 2 of establishing the attention-based CNN-LSTM-BILSTM model comprises, in particular,
step 2.1: and a prediction precision improving step: the convolutional neural network model is used for extracting input data characteristics, and a batch normalization module is arranged between each convolutional layer of the convolutional neural network model, so that the numerical value output by the whole neural network in the middle of each layer is more stable, the deep neural network is easier to converge, the risk of overfitting of the model is reduced, and the network training efficiency is improved;
step 2.2: training attention promoting step: an attention mechanism module is added between the convolutional layer and the long-time memory module, and high-value features are further extracted through the characteristic that the attention mechanism module distributes weights, so that the prediction of the LSTM-BiLSTM in a longer time sequence is more accurate; and predicting the extracted features through two layers of long-time and short-time memory and one layer of bidirectional long-time and short-time memory.
In one embodiment, said step 2.1: the batch normalization formula of the batch normalization module in the prediction precision improving step is as follows:
where xi is the input value for the current layer; yi is the output after batch regularization; m represents the number of samples in a batch; uB is the average of the input values;
is the variance between the input values; epsilon is a smoothing term used to avoid dividing by 0 in the case of very small variance;
is the value after xi batch regularization. The parameters γ and β are obtained by back propagation training.
In one embodiment, the output formula of the attention mechanism module for training the attention boosting step in step 2.2 is as follows:
αijthe weight distributed for the attention mechanism as the characteristic can be obtained through network training; c. CiAn output representing a mechanism of attention; h isjRepresenting a global feature.
In one embodiment, the extracted features are predicted by two layers of long-short term memory and one layer of bidirectional long-short term memory, which specifically,
establishing an LSTM model:
the equations for the different pixels in the LSTM are as follows:
in the above formula, xtRepresenting the t-th time series value input in the LSTM; c. CtRepresenting the current cellular state of the LSTM neurons;
a new candidate for the input forgetting gate generated for tanh; h istRepresenting the hidden state of the LSTM neuron; i.e. it,ft,otRepresenting the t-th input gate, the forgetting gate and the output gate. Wxi,Wxf,Wxo,WxcRespectively representing the weights of the input gate, the forgetting gate, the output gate and the memory unit. Whi,Whf,Who,WhcRepresenting weights from the hidden layer to the input gate, the forgetting gate, the output gate, and the memory cell; bi,bf,bo,bcRespectively representing the offset of the input gate, the forgetting gate, the output gate and the memory cell. Tan h represents a hyperbolic tangent activation function, and an indicates a point-by-point multiplication. In order to obtain the optimal parameters, both CNN and LSTM may use back propagation method to adjust the parameters of the model during training;
establishing a BilSTM model:
bidirectional long-term short-term memory (BilSTM) is a combination of forward LSTM and backward LSTM, and can predict data bidirectionally from the positive and negative directions of time, and then the outputs are connected. LSTM can only model and predict data from the forward direction of time, but modeling only from the forward direction of time ignores information that is reverse in time. BilSTM adds an inverse LSTM, so that BilSTM can capture patterns that LSTM may ignore.
In one embodiment, the training model and the step of predicting the short-term load of the integrated energy system based on the training model may be, in particular,
step 3.1: setting a training loss function as a Root Mean Square Error (RMSE), setting iteration times, and selecting an Adam optimizer to optimize parameters in the network;
step 3.2: inputting the preprocessed data into a model, obtaining a final model after training is finished, and predicting the short-term load of the comprehensive energy system by using the model.
In one embodiment, the Root Mean Square Error (RMSE) formula is:
wherein y isiIs the value predicted by the prediction network,
is the true value, and M is the number of predicted results.
The technical scheme provided by the embodiment of the invention has the following beneficial effects: for a comprehensive energy supply area containing multiple loads, the influences of temperature, cold load, historical electric load data and natural gas load are fully considered, and a CNN-LSTM-BiLSTM load prediction model based on an attention system is obtained by applying CNN, the attention system, LSTM and a BiLSTM model;
preprocessing historical data such as temperature, natural gas load, cold load, electric load and the like, inputting the data into a model for prediction, and obtaining a comprehensive energy system short-term load prediction method based on an attention mechanism;
the method is suitable for short-term load prediction of the comprehensive energy system, can fully mine useful information in historical data, and has better prediction accuracy compared with the traditional method. Provides technical support for the operation optimization of comprehensive energy sources and has great practical value.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a diagram of a CNN-LSTM-BILSTM model architecture based on an attention mechanism according to the present invention;
FIG. 2 is a diagram of the structure of the BilSTM according to the present invention;
FIG. 3 is a schematic representation of the serialization of features according to the present invention;
FIG. 4 is a schematic diagram of a training process of the short-term load forecasting method of the integrated energy system based on the attention mechanism;
FIG. 5 is a schematic diagram of the training results of the short-term load forecasting method of the integrated energy system based on the attention mechanism according to the present invention;
FIG. 6 is a comparison of various models versus a test set daily load prediction;
FIG. 7 is a flowchart illustrating steps of a method for short term load forecasting of an integrated energy system based on an attention mechanism according to the present invention;
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The technical scheme provided by the embodiment of the invention relates to a comprehensive energy system short-term load prediction method based on an attention mechanism, in particular to the field of energy load prediction. In the related art, conventional power system load prediction usually considers weather, economy, electricity price and holiday factors. For the IES, the influence of cold (heat) source, natural gas, solar energy, etc. on the system load needs to be considered. Furthermore, due to the time series and non-linearity of the load, and the power system is greatly affected by the demand side, it is difficult to predict the long-term load of the IES. The short-term load forecast refers to load of each hour or each day after load forecast according to historical data of previous days, and provides theoretical basis for the IES to make a power generation plan. Based on the above, the comprehensive energy system short-term load prediction method based on the attention mechanism provided by the technical scheme of the disclosure considers the influence of temperature, cold load, electric load data and natural gas load on the electric load of the comprehensive energy system, can fully mine useful information in historical data, provides technical support for operation optimization of comprehensive energy, and has great practical value.
Fig. 7 is a flowchart illustrating steps of a method for predicting short-term load of an integrated energy system based on attention mechanism according to the disclosed technical solution.
Step S01 input data preprocessing
The specific process is as follows: historical data such as temperature, cold load, gas consumption, power load and the like of june to september in a certain park are used as input data, feature normalization is carried out on the input data, and errors caused by too large difference among the data are prevented. The data is serialized, the input non-time-series data is generated into time-series characteristics by a sliding window method, the load and the influence factor value at a certain moment are represented by related time-series characteristics, and the historical loads at all moments are represented by the related characteristics together. 80% of the data set obtained after the serialization is used as a training set, and 20% is used as a test set. The data of the training set is used for training the model; and the test set is used for testing the trained model, and the error on the test set is used as the generalization error of the final model in the corresponding real scene to distribute the data proportion to the training set and the test set.
A characteristic normalization formula:
wherein a isiIs the ith data to be normalized, aimax,aiminIs the maximum minimum value before normalization, aniIs the normalized data.
The window sliding method comprises the following steps:
through multi-parameter fine adjustment experiments, when the historical data input for nearly 5 days are selected, the trained model has the highest prediction precision, so that the input is the power load, the temperature, the cold load and the gas consumption per hour for nearly 5 days. The sliding window width is set to record hourly over 5 days and the step size is set to 1. Gradually sliding the window, and collectively representing the historical loads at each time by the 5-day related characteristics thereof, as shown in fig. 3:
step S02: a CNN-LSTM-BILSTM model based on the attention mechanism is established, as shown in FIG. 1.
The specific process is as follows: the CNN is designed into three one-dimensional convolution layers, and the number of the convolution kernels is set to be 16, 32 and 64 respectively. The co nvolutional kernel size is set to 2 and the tapering step is set to 1. Crossing the convolution kernel over the features can extract valid features. After the convolutional layer, a Max Pooling layer was added, the firing window size was set to 2, and the striping step was set to 1. The MaxPholing layer can reduce the complexity of the features and avoid model overfitting. And a batch normalization module is added between each convolution layer of the convolutional neural network, so that the output of each convolution layer is limited in a proper range, and the network training efficiency is improved. And adding an attention mechanism module before the features are input into the LSTM-BilSTM, and further extracting high-value features by the characteristic that the attention mechanism module distributes weights, so that the LSTM-BilSTM can perform better in a longer time sequence. And predicting the extracted characteristics through two layers of long-time and short-time memories and one layer of bidirectional long-time and short-time memories to obtain the power load of the next hour.
(a) Batch normalization formula:
where xi is the input value of the current layer and yi is the output after batch regularization; m is expressed as the number of small batch entries. u. ofBIs the average of the input values and is,
is the variance between the input values, and epsilon is a smoothing term to avoid dividing by 0 in the case of very small variances.
Are values after xi batch regularization the parameters γ and β are obtained by back propagation training. After batch normalization, the output is limited within a proper range, and the training efficiency of the network is greatly improved.
(b) Attention output formula:
αijthe weights assigned to features for attention mechanism can be derived through network training. c. CiIndicating the output of the attention mechanism. h isjRepresenting a global feature. The attention mechanism is divided into soft attention and hard attention, and the present invention adopts soft attention.
(c) Long-short memory (LSTM) and bidirectional long-short memory (BiLSTM) models:
LSTM model:
the LSTM introduces functions of a forgetting gate, an input gate and an output gate to effectively avoid the problems of gradient explosion and gradient disappearance in the RNN training process. Three gate functions can effectively solve the RNN training problem. The memory unit can control the conversion of different time information. The input gate determines the current time to pass on to the next message. The forget gate indicates how much of the information the current neuron retained was delivered by the previous neuron. The output gate determines the output of the current state to the next state. The equations for the different picture elements in the LSTM are as follows.
In the above formula, xtRepresenting the t-th time series value entered in the LSTM. c. CtIndicating the current cellular state of LSTM neurons.
The input generated for tanh forgets the new candidate of the gate. h istIndicating an LSTM neuron hidden state. i.e. it,ft,otRepresenting the t-th input gate, the forgetting gate and the output gate. Wxi,Wxf,Wxo,WxcRespectively representing the weights of the input gate, the forgetting gate, the output gate and the memory unit. Whi,Whf,Who,WhcRepresenting the weights from the hidden layer to the input gate, the forgetting gate, the output gate and the memory cell. bi,bf,bo,bcRespectively representing the offset of the input gate, the forgetting gate, the output gate and the memory cell. Tan h represents a hyperbolic tangent activation function, and an indicates a point-by-point multiplication. To obtain the optimal parameters, either CNN or LSTM can use back-propagation methods to adjust the model parameters during the training process.
BilSTM model:
bidirectional long-term short-term memory (BilSTM) is a combination of forward LSTM and backward LSTM, and can predict data bidirectionally from the positive and negative directions of time, and then the outputs are connected. LSTM can only model and predict data from the forward direction of time, but modeling only from the forward direction of time ignores information that is reverse in time. BilSTM adds an inverse LSTM, so that BilSTM can capture patterns that LSTM may ignore. The structure of BilSTM is shown in FIG. 2.
Step S03: and (5) training and predicting the model.
The specific process is as follows: the training loss function is set to Root Mean Square Error (RMSE), the number of iterations is set to 200, the batch size is set to 128, and the Adam optimizer is selected to optimize parameters in the network. The training process is shown in fig. 4. Inputting the preprocessed data into a model, obtaining a final model after training is finished, and performing short-term load prediction on the comprehensive energy system by using the model, wherein the result is shown in fig. 5.
Root Mean Square Error (RMSE) equation:
wherein y isiIs the value predicted by the prediction network,
is the true value, and M is the number of predicted results.
By combining all the technical schemes, the invention has the advantages and positive effects that: the model is superior to other models in prediction accuracy, after 10 times of repeated calculation, compared with CNN-BilSTM, CNN-LSTM, LSTM, Back Propagation Neural Network (BPNN), Random Forest Regression (RFR) and support vector machine regression (SVR), indexes such as Mean Square Error (MSE), Mean Absolute Error (MAE) and R2 score of the prediction result are compared, and the result is shown as follows.
TABLE 1 comparison of the results of the different models
As shown in FIG. 6, FIG. 6 is a comparison graph of various models versus load prediction of the test set on a day, and from 10 calculations, it can be seen that LSTM-BilSTM is superior to LSTM, which indicates that LSTM-BilSTM is more suitable for load prediction in IES. Furthermore, models using CNN are superior to models that do not, indicating that CNN can efficiently extract and create features to improve the performance of predictive models. Furthermore, the performance of the attention-based CNN-LSTM-BilSTM, CNN-BilSTM, LSTM is superior to RFR and SVR, indicating that the structure of deep learning is superior to the traditional machine learning model. The CNN-L STM-BilSTM model based on attention performs best in the aspects of MAE, RMSE and R2 scores, and the superiority of the model is proved.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When used in whole or in part, can be implemented in a computer program product that includes one or more computer instructions. When loaded or executed on a computer, cause the flow or functions according to embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.)). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. The method for predicting the short-term load of the comprehensive energy system based on the attention mechanism is characterized by comprising the following steps of:
step 1: preprocessing input data;
step 2: establishing a CNN-LSTM-BILSTM model based on an attention system:
and step 3: and training the model and predicting the short-term load of the comprehensive energy system according to the training model.
2. The method for integrated energy system short-term load prediction based on attention mechanism according to claim 1, characterized in that in the input data preprocessing step, which is specifically,
step 1.1: performing characteristic normalization on input data;
step 1.2: serializing the data obtained in step 1.1;
step 1.3: and distributing data proportion to the training set and the test set.
3. The method for forecasting short-term load of an integrated energy system based on an attention mechanism as claimed in claim 1, wherein the step 1.1 utilizes a characteristic normalization formula in the characteristic integration of the input data:
wherein a isiIs the ith data to be normalized, aimax,aiminIs the maximum minimum value before normalization, aniIs the normalized data.
4. The integrated energy system short-term load forecasting method based on attention mechanism as claimed in claim 1, characterized in that the step 1.2: serializing the data obtained in step 1.1, specifically,
generating the input non-time series data into time series characteristics by using a sliding window method, representing the load and the influence factor value at a certain moment by using related time series characteristics, commonly representing the historical load at each moment by using the related characteristics,
the sliding window method, which is specifically,
setting the width of a sliding window as an hourly record in T days and setting the step length by taking the hourly electric load, the temperature, the cooling load and the air consumption of the current time for T days as input characteristics of a model; and gradually sliding a window, and jointly representing the historical load at each moment by the relevant characteristics of T days.
5. The method for forecasting the short-term load of the integrated energy system based on the attention mechanism as claimed in claim 1, wherein the step 2 of establishing the CNN-LSTM-BILSTM model based on the attention mechanism comprises the steps of,
step 2.1: and a prediction precision improving step: the convolutional neural network model is used for extracting input data characteristics, and a batch normalization module is arranged between each convolutional layer of the convolutional neural network model, so that the numerical value output by the whole neural network in the middle of each layer is more stable, the deep neural network is easier to converge, the risk of overfitting of the model is reduced, and the network training efficiency is improved;
step 2.2: training attention promoting step: an attention mechanism module is added between the convolutional layer and the long-time memory module, and high-value features are further extracted through the characteristic that the attention mechanism module distributes weights, so that the prediction of the LSTM-BiLSTM in a longer time sequence is more accurate; and predicting the extracted features through two layers of long-time and short-time memory and one layer of bidirectional long-time and short-time memory.
6. The integrated energy system short-term load forecasting method based on attention mechanism as claimed in claim 1, characterized in that the step 2.1: the batch normalization formula of the batch normalization module in the prediction precision improving step is as follows:
wherein xiIs the input value of the current layer; y isiIs the output after batch regularization;
m represents the number of samples in a batch; uB is the average of the input values;
epsilon is a smoothing term used to avoid dividing by 0 in the case of very small variance;
7. The method for integrated energy system short term load prediction based on attention mechanism of claim 6, characterized in that the output formula of the attention mechanism of the step 2.2 training attention mechanism module of the attention boost step is:
αijis to be notedThe gravity mechanism is a weight distributed to the features and can be obtained through network training; c. CiAn output representing a mechanism of attention; h isjRepresenting a global feature.
8. The method for forecasting the short-term load of the integrated energy system based on the attention mechanism as claimed in claim 6, wherein the extracted features are forecasted by two layers of long-term and short-term memory and one layer of bidirectional long-term and short-term memory, which is specifically,
establishing an LSTM model:
the equations for the different pixels in the LSTM are as follows:
in the above formula, xtRepresenting the t-th time series value input in the LSTM; c. CtRepresenting the current cellular state of the LSTM neurons;
a new candidate for the input forgetting gate generated for tanh; h istRepresenting the hidden state of the LSTM neuron; i.e. it,ft,otRepresenting the t-th input gate, the forgetting gate and the output gate. Wxi,Wxf,Wxo,WxcRespectively representing the weights of the input gate, the forgetting gate, the output gate and the memory unit. Whi,Whf,Who,WhcRepresenting weights from the hidden layer to the input gate, the forgetting gate, the output gate, and the memory cell; bi,bf,bo,bcRespectively representing the offset of the input gate, the forgetting gate, the output gate and the memory cell. Tan h represents a hyperbolic tangent activation function, which indicates a point-by-point multiplication;
establishing a BilSTM model:
bidirectional long-term short-term memory (BilSTM) is a combination of forward LSTM and backward LSTM, and can predict data bidirectionally from the positive and negative directions of time, and then the outputs are connected. LSTM can only model and predict data from the forward direction of time, but modeling only from the forward direction of time ignores information that is reverse in time. BilSTM adds an inverse LSTM, so that BilSTM can capture patterns that LSTM may ignore.
9. The attention mechanism-based short-term load forecasting method of an integrated energy system according to claim 1, wherein in the step of training the model and forecasting the short-term load of the integrated energy system based on the training model, specifically,
step 3.1: setting a training loss function as a Root Mean Square Error (RMSE), setting iteration times, and selecting an Adam optimizer to optimize parameters in the network;
step 3.2: inputting the preprocessed data into a model, obtaining a final model after training is finished, and predicting the short-term load of the comprehensive energy system by using the model.
10. The integrated energy system short-term load prediction method based on attention mechanism of claim 1, characterized in that the Root Mean Square Error (RMSE) formula is:
wherein y isiIs the value predicted by the prediction network,
is the true value, and M is the number of predicted results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011410725.XA CN112529283A (en) | 2020-12-04 | 2020-12-04 | Comprehensive energy system short-term load prediction method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011410725.XA CN112529283A (en) | 2020-12-04 | 2020-12-04 | Comprehensive energy system short-term load prediction method based on attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112529283A true CN112529283A (en) | 2021-03-19 |
Family
ID=74997694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011410725.XA Pending CN112529283A (en) | 2020-12-04 | 2020-12-04 | Comprehensive energy system short-term load prediction method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112529283A (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906673A (en) * | 2021-04-09 | 2021-06-04 | 河北工业大学 | Lower limb movement intention prediction method based on attention mechanism |
| CN113095598A (en) * | 2021-05-07 | 2021-07-09 | 国网山东省电力公司经济技术研究院 | Multi-energy load prediction method, system, device and medium |
| CN113177366A (en) * | 2021-05-28 | 2021-07-27 | 华北电力大学 | Comprehensive energy system planning method and device and terminal equipment |
| CN113177666A (en) * | 2021-05-21 | 2021-07-27 | 同济大学 | Prediction method based on non-invasive attention preprocessing process and BilSTM model |
| CN113206756A (en) * | 2021-04-22 | 2021-08-03 | 大连大学 | Network flow prediction method based on combined model |
| CN113393121A (en) * | 2021-06-15 | 2021-09-14 | 贵州电网有限责任公司 | Non-invasive load identification method based on load power fingerprint characteristics |
| CN113435663A (en) * | 2021-07-15 | 2021-09-24 | 国网冀北电力有限公司唐山供电公司 | CNN-LSTM combined load prediction method considering electric vehicle charging load influence |
| CN113537459A (en) * | 2021-06-28 | 2021-10-22 | 淮阴工学院 | Method for predicting humiture of drug storage room |
| CN113610277A (en) * | 2021-07-15 | 2021-11-05 | 中山大学 | Power load prediction method, device and medium based on space-time attention mechanism |
| CN113762601A (en) * | 2021-08-12 | 2021-12-07 | 北京市燃气集团有限责任公司 | Gas user gas consumption prediction method and device |
| CN113822482A (en) * | 2021-09-24 | 2021-12-21 | 内蒙古电力(集团)有限责任公司内蒙古电力科学研究院分公司 | Method and device for establishing load prediction model of comprehensive energy system |
| CN113837464A (en) * | 2021-09-22 | 2021-12-24 | 浙大城市学院 | Load prediction method of cogeneration boiler based on CNN-LSTM-Attention |
| CN113837443A (en) * | 2021-08-26 | 2021-12-24 | 国网河北省电力有限公司邢台供电分公司 | Transformer substation line load prediction method based on depth BilSTM |
| CN114187059A (en) * | 2021-12-27 | 2022-03-15 | 广东工业大学 | Short-term electricity price prediction method |
| CN114239905A (en) * | 2021-10-29 | 2022-03-25 | 国网天津市电力公司 | ResNet-LSTM-based power distribution station area short-term load prediction method and device |
| CN115481788A (en) * | 2022-08-31 | 2022-12-16 | 北京建筑大学 | Load prediction method and system for phase change energy storage system |
| CN115545354A (en) * | 2022-11-30 | 2022-12-30 | 国网天津市电力公司信息通信公司 | Artificial intelligence load prediction method and system for intelligent Internet of things platform |
| CN115587617A (en) * | 2022-10-17 | 2023-01-10 | 国网黑龙江省电力有限公司 | Method and device for capturing sequence information of long-time sequence and method and device for predicting short-term power consumption |
| CN115689068A (en) * | 2023-01-01 | 2023-02-03 | 北京云庐科技有限公司 | Thermal load prediction method based on room temperature correction and CNN-BilSTM |
| CN115936185A (en) * | 2022-11-15 | 2023-04-07 | 国网江苏省电力有限公司苏州供电分公司 | Short-term power load and carbon emission prediction method and system based on DCNN-LSTM-AE-AM |
| CN116090359A (en) * | 2023-04-11 | 2023-05-09 | 南京师范大学 | Comprehensive energy system multi-element load prediction method based on combined prediction model |
| CN116361709A (en) * | 2023-03-31 | 2023-06-30 | 山东省计算中心（国家超级计算济南中心） | Self-adaptive power load identification method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110298501A (en) * | 2019-06-21 | 2019-10-01 | 河海大学常州校区 | Electric load prediction technique based on long Memory Neural Networks in short-term |
| WO2020077232A1 (en) * | 2018-10-12 | 2020-04-16 | Cambridge Cancer Genomics Limited | Methods and systems for nucleic acid variant detection and analysis |
| CN111191841A (en) * | 2019-12-30 | 2020-05-22 | 润联软件系统(深圳)有限公司 | Power load prediction method and device, computer equipment and storage medium |
-
2020
- 2020-12-04 CN CN202011410725.XA patent/CN112529283A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020077232A1 (en) * | 2018-10-12 | 2020-04-16 | Cambridge Cancer Genomics Limited | Methods and systems for nucleic acid variant detection and analysis |
| CN110298501A (en) * | 2019-06-21 | 2019-10-01 | 河海大学常州校区 | Electric load prediction technique based on long Memory Neural Networks in short-term |
| CN111191841A (en) * | 2019-12-30 | 2020-05-22 | 润联软件系统(深圳)有限公司 | Power load prediction method and device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| KUIHUA WU等: "An attention-based CNN-LSTM-BiLS TM model for shortterm electric load forecasting in integrated energy system", 《WILEY》 * |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906673A (en) * | 2021-04-09 | 2021-06-04 | 河北工业大学 | Lower limb movement intention prediction method based on attention mechanism |
| CN113206756A (en) * | 2021-04-22 | 2021-08-03 | 大连大学 | Network flow prediction method based on combined model |
| CN113206756B (en) * | 2021-04-22 | 2023-10-03 | 大连大学 | Network flow prediction method based on combined model |
| CN113095598A (en) * | 2021-05-07 | 2021-07-09 | 国网山东省电力公司经济技术研究院 | Multi-energy load prediction method, system, device and medium |
| CN113177666A (en) * | 2021-05-21 | 2021-07-27 | 同济大学 | Prediction method based on non-invasive attention preprocessing process and BilSTM model |
| WO2022241932A1 (en) * | 2021-05-21 | 2022-11-24 | 同济大学 | Prediction method based on non-intrusive attention preprocessing process and bilstm model |
| CN113177366A (en) * | 2021-05-28 | 2021-07-27 | 华北电力大学 | Comprehensive energy system planning method and device and terminal equipment |
| CN113177366B (en) * | 2021-05-28 | 2024-02-02 | 华北电力大学 | Comprehensive energy system planning method and device and terminal equipment |
| CN113393121A (en) * | 2021-06-15 | 2021-09-14 | 贵州电网有限责任公司 | Non-invasive load identification method based on load power fingerprint characteristics |
| CN113537459B (en) * | 2021-06-28 | 2024-04-26 | 淮阴工学院 | Drug warehouse temperature and humidity prediction method |
| CN113537459A (en) * | 2021-06-28 | 2021-10-22 | 淮阴工学院 | Method for predicting humiture of drug storage room |
| CN113610277A (en) * | 2021-07-15 | 2021-11-05 | 中山大学 | Power load prediction method, device and medium based on space-time attention mechanism |
| CN113435663A (en) * | 2021-07-15 | 2021-09-24 | 国网冀北电力有限公司唐山供电公司 | CNN-LSTM combined load prediction method considering electric vehicle charging load influence |
| CN113762601A (en) * | 2021-08-12 | 2021-12-07 | 北京市燃气集团有限责任公司 | Gas user gas consumption prediction method and device |
| CN113837443A (en) * | 2021-08-26 | 2021-12-24 | 国网河北省电力有限公司邢台供电分公司 | Transformer substation line load prediction method based on depth BilSTM |
| CN113837443B (en) * | 2021-08-26 | 2023-12-05 | 国网河北省电力有限公司邢台供电分公司 | Substation line load prediction method based on depth BiLSTM |
| CN113837464A (en) * | 2021-09-22 | 2021-12-24 | 浙大城市学院 | Load prediction method of cogeneration boiler based on CNN-LSTM-Attention |
| CN113822482A (en) * | 2021-09-24 | 2021-12-21 | 内蒙古电力(集团)有限责任公司内蒙古电力科学研究院分公司 | Method and device for establishing load prediction model of comprehensive energy system |
| CN114239905A (en) * | 2021-10-29 | 2022-03-25 | 国网天津市电力公司 | ResNet-LSTM-based power distribution station area short-term load prediction method and device |
| CN114187059A (en) * | 2021-12-27 | 2022-03-15 | 广东工业大学 | Short-term electricity price prediction method |
| CN115481788B (en) * | 2022-08-31 | 2023-08-25 | 北京建筑大学 | Phase change energy storage system load prediction method and system |
| CN115481788A (en) * | 2022-08-31 | 2022-12-16 | 北京建筑大学 | Load prediction method and system for phase change energy storage system |
| CN115587617A (en) * | 2022-10-17 | 2023-01-10 | 国网黑龙江省电力有限公司 | Method and device for capturing sequence information of long-time sequence and method and device for predicting short-term power consumption |
| CN115936185A (en) * | 2022-11-15 | 2023-04-07 | 国网江苏省电力有限公司苏州供电分公司 | Short-term power load and carbon emission prediction method and system based on DCNN-LSTM-AE-AM |
| CN115545354A (en) * | 2022-11-30 | 2022-12-30 | 国网天津市电力公司信息通信公司 | Artificial intelligence load prediction method and system for intelligent Internet of things platform |
| CN115689068A (en) * | 2023-01-01 | 2023-02-03 | 北京云庐科技有限公司 | Thermal load prediction method based on room temperature correction and CNN-BilSTM |
| CN115689068B (en) * | 2023-01-01 | 2024-01-26 | 北京云庐科技有限公司 | Thermal load prediction method based on room temperature correction and CNN-BiLSTM |
| CN116361709A (en) * | 2023-03-31 | 2023-06-30 | 山东省计算中心（国家超级计算济南中心） | Self-adaptive power load identification method and device |
| CN116361709B (en) * | 2023-03-31 | 2023-10-31 | 山东省计算中心（国家超级计算济南中心） | Self-adaptive power load identification method and device |
| CN116090359A (en) * | 2023-04-11 | 2023-05-09 | 南京师范大学 | Comprehensive energy system multi-element load prediction method based on combined prediction model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112529283A (en) | Comprehensive energy system short-term load prediction method based on attention mechanism | |
| Wang et al. | Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting | |
| Gao et al. | Deep learning and transfer learning models of energy consumption forecasting for a building with poor information data | |
| Ghaderi et al. | Deep forecast: Deep learning-based spatio-temporal forecasting | |
| Xiang et al. | Ultra-short term wind power prediction applying a novel model named SATCN-LSTM | |
| Ma et al. | A hybrid attention-based deep learning approach for wind power prediction | |
| Lee et al. | PV power prediction in a peak zone using recurrent neural networks in the absence of future meteorological information | |
| Li et al. | A hybrid annual power load forecasting model based on generalized regression neural network with fruit fly optimization algorithm | |
| Huang et al. | A control strategy based on deep reinforcement learning under the combined wind-solar storage system | |
| CN104636985A (en) | Method for predicting radio disturbance of electric transmission line by using improved BP (back propagation) neural network | |
| CN112163689A (en) | Short-term load quantile probability prediction method based on depth Attention-LSTM | |
| CN112116144A (en) | Regional power distribution network short-term load prediction method | |
| Liu et al. | Heating load forecasting for combined heat and power plants via strand-based LSTM | |
| CN113158572A (en) | Short-term load prediction method and device | |
| CN112488452B (en) | Energy system management multi-time scale optimal decision method based on deep reinforcement learning | |
| Sodsong et al. | Short-term solar PV forecasting using gated recurrent unit with a cascade model | |
| Islam et al. | LSTM-based electrical load forecasting for Chattogram city of Bangladesh | |
| Shang et al. | Decomposition-based wind speed forecasting model using causal convolutional network and attention mechanism | |
| CN115115125A (en) | Photovoltaic power interval probability prediction method based on deep learning fusion model | |
| Parizad et al. | Deep learning algorithms and parallel distributed computing techniques for high-resolution load forecasting applying hyperparameter optimization | |
| Joudaki | Long-Term Wind Speed and Power Forecasting Based on LSTM: A Comprehensive Study | |
| CN112330044A (en) | Support vector regression model based on iterative aggregation grid search algorithm | |
| Du et al. | An innovative interpretable combined learning model for wind speed forecasting | |
| Liu et al. | A hybrid deep learning model based on parallel architecture TCN-LSTM with Savitzky-Golay filter for wind power prediction | |
| CN116402194A (en) | Multi-time scale load prediction method based on hybrid neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210319 |