CN116843057A - Wind power ultra-short-term prediction method based on LSTM-ViT - Google Patents
Wind power ultra-short-term prediction method based on LSTM-ViT Download PDFInfo
- Publication number
- CN116843057A CN116843057A CN202310632241.7A CN202310632241A CN116843057A CN 116843057 A CN116843057 A CN 116843057A CN 202310632241 A CN202310632241 A CN 202310632241A CN 116843057 A CN116843057 A CN 116843057A
- Authority
- CN
- China
- Prior art keywords
- data
- wind power
- vit
- lstm
- short
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012549 training Methods 0.000 claims abstract description 20
- 238000010606 normalization Methods 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000012795 verification Methods 0.000 claims abstract description 8
- 238000012360 testing method Methods 0.000 claims abstract description 6
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000007246 mechanism Effects 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 20
- 230000004913 activation Effects 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 16
- 230000009466 transformation Effects 0.000 claims description 14
- 230000001413 cellular effect Effects 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000000844 transformation Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 abstract description 5
- 230000000007 visual effect Effects 0.000 abstract description 4
- 238000011161 development Methods 0.000 description 4
- 238000010248 power generation Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 229910052799 carbon Inorganic materials 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000002401 inhibitory effect Effects 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000005728 strengthening Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000006386 neutralization reaction Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J3/00—Circuit arrangements for ac mains or ac distribution networks
- H02J3/004—Generation forecast, e.g. methods or systems for forecasting future energy generation
-
- H—ELECTRICITY
- H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
- H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
- H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
- H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Water Supply & Treatment (AREA)
- Public Health (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Primary Health Care (AREA)
- Quality & Reliability (AREA)
- Power Engineering (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a wind power ultra-short-term prediction method based on LSTM-ViT, which comprises the following steps of: s1, data preprocessing: s1-1, dividing meteorological data into four data sets according to quarters, wherein each data set comprises meteorological data and wind power data; s1-2, carrying out normalization processing on meteorological data and wind power data; s1-3, dividing data in each data set into a training set, a verification set and a test set; s2, building an LSTM-ViT wind power ultra-short-term prediction model; s3, training a wind power ultra-short-term prediction model; s4, predicting wind power. The method for predicting the ultra-short-term wind power by combining the long-short-term memory network and the visual self-attention model can effectively extract the time characteristics and the relevant characteristics among the meteorological factors, and establish the complex nonlinear relation between the meteorological factors and the wind power, so that the ultra-short-term wind power can be accurately predicted, and the safe and stable operation of a power system is ensured.
Description
Technical Field
The invention belongs to the technical field of wind power generation, and particularly relates to a wind power ultra-short-term prediction method based on LSTM-ViT.
Background
With global shortage of fossil energy and demand for energy conservation and emission reduction, fossil energy has been gradually replaced with clean energy. Wind energy has attracted considerable attention worldwide as a clean, renewable energy source. Through decades of development, wind power generation has become an energy source with mature technology, low development cost and the most development prospect. In 2020, china proposes the strategic targets of 'carbon reaching peak before 2030 and carbon neutralization before 2060', and promotes the rapid development of renewable energy sources such as wind energy and the like. However, the characteristics of the wind farm such as volatility, nonlinearity, randomness and uncertainty bring serious challenges to safe and stable operation of the wind farm, and wind power prediction is a key to effectively scheduling wind power resources.
In order to reduce the adverse effect of wind power uncertainty on the operation of a power system, a plurality of students study a wind power prediction model to improve the prediction accuracy of the wind power prediction model. Traditional machine learning algorithms are easy to implement but difficult to express complex nonlinear relations between inputs and outputs, and limit prediction accuracy. In recent years, deep learning algorithms are rapidly developed, and how to apply the deep learning algorithms to accurately predict wind power is a subject faced by relevant technicians.
In the prior art, the patent number is CN202210441115.9, the publication number is CN115034432A, and the patent name is the invention patent of a wind power generation set wind speed prediction method of a wind power plant, which discloses a technical scheme constructed by combining the spatial distribution characteristics of the wind power generation set and the time characteristics of wind speed data, so that the wind speed of the whole plant set is effectively predicted. However, the method is difficult to realize ultra-short-term prediction of wind power on the premise of ensuring prediction accuracy, so that more exploration is required for relevant technicians in the direction of the ultra-short-term prediction method of wind power.
Disclosure of Invention
The invention provides a wind power ultra-short-term prediction method based on LSTM-ViT for solving the technical problems in the prior art, and the method is based on long-short-term memory network (long-short term memory, LSTM) and visual self-attention model (vision transformer, viT) so as to improve the prediction precision of wind power and ensure the safe and stable operation of a power system.
The invention comprises the following technical scheme:
a wind power ultra-short-term prediction method based on LSTM-ViT comprises the following steps:
s1, data preprocessing:
s1-1, dividing weather data of at least one year required for predicting wind power into four data sets according to quarters, wherein the data in each data set comprises weather data and corresponding wind power data;
s1-2, carrying out normalization processing on meteorological data and wind power data;
s1-3, respectively dividing the data in each data set into a training set, a verification set and a test set;
s2, building a wind power prediction model:
performing feature extraction on meteorological data by utilizing LSTM, establishing a relation between the extracted features and output wind power by utilizing ViT, and establishing an LSTM-ViT wind power ultra-short-term prediction model;
s3, training a wind power ultra-short-term prediction model:
inputting training set data into an LSTM-ViT wind power ultra-short-term prediction model, training the model, and adjusting network parameters of LSTM and ViT by using a verification set in the training process;
s4, predicting wind power:
and (3) carrying out normalization processing on the meteorological data acquired in real time, and inputting the meteorological data into a trained LSTM-ViT wind power ultra-short-term prediction model to realize the prediction of wind power at the next moment.
Further, the LSTM is configured to establish a nonlinear relationship between the extracted feature and the wind power output, where the input is normalized meteorological data, and the output is a numerical value after feature extraction. The LSTM in the S2 comprises an input gate, a forget gate and an output gate, wherein the input gate controls input information to be sent into the memory cell, the output gate controls information output of the memory cell, next hidden state information is determined, and the forget gate determines which information needs to be deleted and forgotten. The calculation process in the LSTM unit is as follows:
f t ＝Sigmoid(ω f ×[h t-1 ,I t ]+β f )
i t ＝Sigmoid(ω i ×[h t-1 ,I t ]+βi)
y t ＝Sigmoid(ω o ×[h t-1 ,I t ]+β o )
h t ＝y t tanh(H t )
wherein ,ft A forgetting door at the time t; the sigmoid and tanh functions are activation functions; h is a t-1 Is the data output information of time t-1; i t Is the data input information at time t. Omega f ,ω i ,ω c ,ω o Is a weight coefficient; beta f ,β i ,β c ,β o Is a bias parameter; i.e t Andan input representing a time t; h t-1 Is the cellular state at time t-1; h t Is the cellular state at time t; h is a t Is the data output information of time t; y is t Is the activation function of time tThe number Sigmoid is output after activation.
Further, the basic structure of the ViT part in S2 is a transducer algorithm, and based on the encoder-decoder structure, a self-attention mechanism (the self-attention mechanism is a core module of the transducer model and is also a part of the encoding module) is introduced, and the calculation formula of the self-attention mechanism is as follows:
wherein Q, K, V is matrix composed of vectors obtained by different linear transformations of input data, softmax (g) is activation function for normalization, d k Is the dimension of K.
Further, the ViT part adopts a multi-head attention mechanism (i.e. uses multiple self-attention mechanisms to calculate so as to capture deeper feature information), and performs stitching and combining on the feature information captured by the multiple self-attention mechanisms to obtain a final feature result, where the calculation formula is as follows:
head i ＝Attention(QW i Q ,KW i K ,VW i V )
MultiHead(Q,K,V)＝Concat(head 1 ,…,head h )W O
wherein ,WQ 、W K、 and WV The dimensions of all three quantities are [ d model ,d K ]；W O Is [ hd ] V ,d model ]。
Furthermore, the ViT part mainly performs matrix multiplication (scaling dot product attention) in the internal structure of the multi-head attention, namely performs linear transformation, and the learning ability of the multi-head attention is not as linear transformation, so that the output of the multi-head attention can perform nonlinear mapping by adding an activation function, strengthening a large part and inhibiting a small part; each layer of the encoder except for the multi-head attention layer comprises a fully-connected feedforward neural network, wherein the fully-connected feedforward neural network consists of two linear transformations, the middle is GeLU activation, and the calculation formula is as follows:
wherein W1 ,W 2 ,b 1 ,b 2 The weights and deviations of the two linear transformation layers, respectively.
Further, the ViT part transform encoder is to repeatedly stack the encoder blocks N times. The transducer is formed by superposing a plurality of transducer blocks. Each transducer block consists of a multi-headed self-attention module and a feedforward neural network module, both of which are provided with a preposed normalization layer and are connected with the output through residual connection. The calculation formula is as follows:
wherein , and />The outputs of the multihead self-attention module and the feedforward neural network module, respectively.
Further, the LSTM-ViT wind power ultra-short-term prediction model comprises an LSTM layer and ViT layers, the time step number of the LSTM layer is 10, the batch size is 64, and the layer number is 1; the number of hidden layers in the ViT layer is 256, the number of multi-head attention mechanisms is 4, and the number of layers is 4, and the multi-head attention mechanism comprises two normalization layers, one multi-head attention mechanism and one feedforward neural network layer; the number of iterations of training was 130 and the batch size was 256.
Further, the meteorological data includes wind speed, wind direction, temperature, air density and air pressure.
Further, the specific steps of normalizing the meteorological data and the wind power data in the step S1-2 are as follows:
by x 1 Representing power data, x 1 ' represents normalized power data, then
in the formula Cwt The installed capacity of the wind farm;
by x 2 Data representing wind speed, temperature, air pressure or air density, x 2 ' represents normalized wind speed, temperature, air pressure or air density data, then
in the formula x2max and x2min Maximum and minimum values of wind speed, temperature, air pressure or air density data respectively;
by x 3 Represents wind direction data, x 3 ' represents normalized wind direction data, then
x 3 '＝sin(x 3 )
The invention has the advantages and positive effects that:
1. the invention utilizes LSTM to extract the time sequence characteristics of meteorological factors and the related characteristics among variables, which is beneficial to improving the characteristic extraction capacity of the model.
2. According to the wind power prediction method, viT is utilized to predict wind power, the relation between the characteristics and the wind power can be well represented, a deep learning model is quickly trained, and accurate wind power prediction is facilitated.
3. The method for predicting the ultra-short-term wind power by combining the long-short-term memory network and the visual self-attention model can effectively extract the time characteristics and the relevant characteristics among the meteorological factors, and establish the complex nonlinear relation between the meteorological factors and the wind power, so that the ultra-short-term wind power can be accurately predicted, and the safe and stable operation of a power system is ensured.
4. The ultra-short-term wind power prediction adopted by the invention can achieve higher prediction precision, and effectively extract the characteristic information of the air-condition factors, thereby comprehensively and accurately predicting the ultra-short-term wind power.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a block diagram of a predictive model;
FIG. 3 is a diagram of the structure of ViT;
FIG. 4 is a block diagram of an LSTM cell;
FIG. 5 is a graph of wind power prediction results at four quarters according to the present invention.
In the figure, viT is a visual self-attention model, LSTM is a long-short time memory network, vi is vision, and T is a self-attention model;
x 1 representing power data, x' 1 Representing normalized power data;
C wt the installed capacity of the wind farm;
x 2 data representing wind speed, temperature, air pressure or air density, x' 2 Representing normalized wind speed, temperature, air pressure or air density data, x 2max and x2min Maximum and minimum values of wind speed, temperature, air pressure or air density data respectively;
x 3 representing wind direction data, x' 3 Representing normalized wind direction data;
an activity (·) is an Activation function; q, K, V is a matrix formed by vectors obtained by different linear transformations of the input data; softmax (g) is the normalized activation function, d k Dimension of K, W Q 、W K、 and WV The dimensions of all three quantities are [ d model ,d K ]，W O Is [ hd ] V ,d model ],f t A forgetting door at the time t;
the sigmoid and tanh functions are activation functions; h is a t-1 Time of yesData output information of interval t-1; i t Is the data input information at time t; omega f 、ω i 、ω c 、ω o Is a weight coefficient; beta f 、β i 、β c 、β o Is a bias parameter; i.e t Andan input representing a time t;
H t-1 is the cellular state at time t-1; h t Is the cellular state at time t; h is a t Is the data output information of time t; y is t Is the output of the activated function Sigmoid at time t.
Detailed Description
In order to further disclose the inventive aspects, features and advantages of the present invention, the following examples are set forth in detail below with reference to the accompanying drawings.
Examples: referring to fig. 1-5, the wind power ultra-short-term prediction method based on LSTM-ViT can effectively extract meteorological factor characteristics and further realize accurate prediction of wind power. The method comprises the following steps:
s1, data preprocessing:
s1-1, dividing weather data of at least one year required for predicting wind power into four data sets according to quarters, wherein the data in each data set comprises weather data and corresponding wind power data; the meteorological data comprise wind speed, wind direction, temperature, air density and air pressure;
s1-2, carrying out normalization processing on meteorological data and wind power data to ensure that the running speed of a model is faster and the convergence is better; the specific method for carrying out normalization processing on the meteorological data and the wind power data comprises the following steps:
by x 1 Representing power data, x 1 ' represents normalized power data, then
in the formula Cwt The installed capacity of the wind farm;
by x 2 Data representing wind speed, temperature, air pressure or air density, x 2 ' represents normalized wind speed, temperature, air pressure or air density data, then
in the formula x2max and x2min Maximum and minimum values of wind speed, temperature, air pressure or air density data respectively;
by x 3 Represents wind direction data, x 3 ' represents normalized wind direction data, then x 3 '＝sin(x 3 )；
S1-3, dividing data in each data set into a training set, a verification set and a test set according to the proportion of 80%, 10% and 10%; the test set is used for being input into the verification set loss minimum model to obtain a prediction result of the test data.
S2, building a wind power prediction model: performing feature extraction on meteorological data by utilizing LSTM, establishing a relation between the extracted features and output wind power by utilizing ViT, and establishing an LSTM-ViT wind power ultra-short-term prediction model;
the LSTM unit structure is shown in fig. 4, and the LSTM is used for establishing a nonlinear relation between the extracted characteristics and the wind power output, wherein the input is normalized meteorological data, and the output is a numerical value after the characteristics are extracted. And the LSTM is utilized to extract the time sequence characteristics of the meteorological factors and the related characteristics among the variables, so that the characteristic extraction capability of the model is improved. The LSTM in the S2 comprises an input gate, a forget gate and an output gate, wherein the input gate controls input information to be sent into the memory cell, the output gate controls information output of the memory cell, next hidden state information is determined, and the forget gate determines which information needs to be deleted and forgotten.
The calculation process in the LSTM unit is as follows:
f t ＝Sigmoid(ω f ×[h t-1 ,I t ]+β f )
i t ＝Sigmoid(ω i ×[h t-1 ,I t ]+βi)
y t ＝Sigmoid(ω o ×[h t-1 ,I t ]+β o )
h t ＝y t tanh(H t )
wherein ,ft A forgetting door at the time t; the sigmoid and tanh functions are activation functions; h is a t-1 Is the data output information of time t-1; i t Is the data input information at time t. Omega f ,ω i ,ω c ,ω o Is a weight coefficient; beta f ,β i ,β c ,β o Is a bias parameter; i.e t Andan input representing a time t; h t-1 Is the cellular state at time t-1; h t Is the cellular state at time t; h is a t Is the data output information of time t; y is t Is the output of the activated function Sigmoid at time t.
ViT the structure is shown in fig. 3, and ViT is utilized to predict wind power, so that the relation between the characteristics and the wind power can be well represented, and the deep learning model can be quickly trained, thereby being beneficial to accurately predicting the wind power. The basic structure of the ViT part in S2 is a transducer algorithm, based on the encoder-decoder structure, and introduces a self-attention mechanism (the self-attention mechanism is a core module of the transducer model and is also a part of the encoding module), and the calculation formula of the self-attention mechanism is as follows:
wherein Q, K, V is matrix composed of vectors obtained by different linear transformations of input data, softmax (g) is activation function for normalization, d k Is the dimension of K.
The multi-head attention mechanism is calculated by using a plurality of self-attention mechanisms so as to capture deeper characteristic information. And splicing and combining the characteristic information captured by the self-attention mechanisms to obtain a final characteristic result, wherein a calculation formula is as follows.
head i ＝Attention(QW i Q ,KW i K ,VW i V )
MultiHead(Q,K,V)＝Concat(head 1 ,…,head h )W O
wherein ,WQ 、W K、 and WV The dimensions of all three quantities are [ d model ,d K ]。W O Is [ hd ] V ,d model ]。
The ViT part mainly performs matrix multiplication (scaling dot product attention) in the internal structure of the multi-head attention, namely performs linear transformation, and the learning capacity of the multi-head attention is not as same as that of nonlinear transformation, so that the output of the multi-head attention can perform nonlinear mapping by adding an activation function, strengthening a large part and inhibiting a small part.
Each layer of the encoder contains a fully connected feedforward neural network, except for the multi-head attention layer. The fully-connected feedforward neural network consists of two linear transformation layers, with GeLU activation in the middle. The calculation formula is as follows:
wherein W1 ,W 2 ,b 1 ,b 2 The weights and deviations of the two linear transformation layers, respectively.
The ViT partial transducer encoder is to repeatedly stack encoder blocks N times. The transducer is formed by superposing a plurality of transducer blocks. Each transducer block consists of a multi-headed self-attention module and a feedforward neural network module, both of which are provided with a preposed normalization layer and are connected with the output through residual connection. The calculation formula is shown below.
wherein , and />The outputs of the multihead self-attention module and the feedforward neural network module, respectively.
The built LSTM-ViT wind power ultra-short-term prediction model structure is shown in fig. 2, and comprises an LSTM layer and a ViT layer (the ViT layer comprises two normalization layers, a multi-head attention mechanism and a feedforward neural network layer). By establishing a complex nonlinear relation between meteorological factors and wind power, the ultra-short-term wind power can be accurately predicted, and safe and stable operation of a power system is ensured. The LSTM layer has time steps of 10, batch size of 64 and layer number of 1; the number of hidden layers in ViT layers is 256, the number of multi-head attention mechanisms is 4, the number of layers is 4, the number of training iterations is 130, and the batch size is 256.
S3, training a wind power ultra-short-term prediction model:
inputting training set data into an LSTM-ViT wind power ultra-short-term prediction model, training the model, and adjusting network parameters of LSTM and ViT by using a verification set in the training process; and verifying the optimal parameter model with the minimum error.
S4, predicting wind power:
and (3) carrying out normalization processing on the meteorological data acquired in real time, and inputting the meteorological data into a trained LSTM-ViT wind power ultra-short-term prediction model to realize the prediction of wind power at the next moment. The LSTM extracts relevant characteristics between time characteristics and variables of meteorological factors, and then ViT establishes a connection between the extracted characteristics and output wind power, namely, the extracted characteristic tensor is processed into a one-dimensional vector which is input into a normalization layer, a multi-head attention mechanism layer and a feedforward neural network layer in ViT, and finally wind power predication values at the next moment are output, so that higher predication precision can be achieved, characteristic information of the meteorological factors can be effectively extracted, and accordingly ultra-short-term wind power can be comprehensively and accurately predicted.
The effectiveness of the present invention is verified by analyzing data of a certain wind farm throughout the year.
The invention is verified by adopting annual meteorological data of a certain wind farm 2010 including wind speed, wind direction, temperature, air density, air pressure and wind power data. The four quarter data statistics of the wind power are shown in table 1. It can be seen from table 1 that the wind farm has different wind power statistics from quarter to quarter.
The data is preprocessed and the model is built and predicted through the steps, and LSTM, viT, CNN-ViT and LSTM-ViT are adopted as comparison models to compare with the effects of the invention.
The prediction performance of the model is evaluated by adopting three evaluation indexes of MAE, RMSE and SMAPE, and the calculation formula is as follows:
wherein N is the total number of predicted points,y i respectively, predicted and actual wind power values.
The results of the evaluation index of each model at four seasons are shown in table 2. As can be seen from Table 2, the wind power ultra-short-term prediction method based on LSTM-ViT provided by the invention can better establish a complex nonlinear relation between meteorological factors and wind power compared with a comparison model, and has higher prediction precision.
Taking the power prediction result of the second quarter as an example, the MAE index of the method provided by the invention is reduced by 35.00%, 32.37% and 5.95% compared with other models, the RMSE index is improved by 23.64%, 14.31% and 0.49% compared with other models, and simultaneously, the SMAPE index is respectively reduced by 28.02%, 10.26% and 20.62% compared with LSTM, viT and CNN-ViT. According to the method, wind power prediction results randomly selected in different seasons are shown in fig. 5, a prediction value curve and an actual value curve are respectively drawn by taking 5min as time resolution, the power prediction results are found to be more consistent with the actual power curve, and the method can track the power change trend, so that higher prediction precision is achieved.
Although the preferred embodiments of the present invention have been described, the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the appended claims. All of which are within the scope of the present invention. The attached table:
table 1 annual wind power data statistics
Table 2 comparison of the prediction results for different models
Claims (10)
1. The wind power ultra-short-term prediction method based on LSTM-ViT is characterized by comprising the following steps of:
s1, data preprocessing: s1-1, dividing weather data of at least one year required for predicting wind power into four data sets according to quarters, wherein the data in each data set comprises weather data and corresponding wind power data; s1-2, carrying out normalization processing on meteorological data and wind power data; s1-3, respectively dividing the data in each data set into a training set, a verification set and a test set;
s2, building a wind power prediction model: performing feature extraction on meteorological data by utilizing LSTM, establishing a relation between the extracted features and output wind power by utilizing ViT, and establishing an LSTM-ViT wind power ultra-short-term prediction model;
s3, training a wind power ultra-short-term prediction model: inputting training set data into an LSTM-ViT wind power ultra-short-term prediction model, training the model, and adjusting network parameters of LSTM and ViT by using a verification set in the training process;
s4, predicting wind power: and (3) carrying out normalization processing on the meteorological data acquired in real time, and inputting the meteorological data into a trained LSTM-ViT wind power ultra-short-term prediction model to realize wind power prediction at the next moment.
2. The ultra-short-term prediction method for wind power based on LSTM-ViT as defined in claim 1, wherein the method comprises the following steps of: the LSTM in the S2 comprises an input gate, a forget gate and an output gate, and the calculation process in the LSTM unit is as follows:
f t ＝Sigmoid(ω f ×[h t-1 ,I t ]+β f )
i t ＝Sigmoid(ω i ×[h t-1 ,I t ]+βi)
y t ＝Sigmoid(ω o ×[h t-1 ,I t ]+β o )
h t ＝y t tanh(H t )
wherein ,ft A forgetting door at the time t; the sigmoid and tanh functions are activation functions; h is a t-1 Is the data output information of time t-1; i t Is the data input information at time t. Omega f ,ω i ,ω c ,ω o Is a weight coefficient; beta f ,β i ,β c ,β o Is a bias parameter; i.e t Andan input representing a time t; h t-1 Is the cellular state at time t-1; h t Is the cellular state at time t; h is a t Is the data output information of time t; y is t Is the output of the activated function Sigmoid at time t.
3. The ultra-short-term prediction method for wind power based on LSTM-ViT as defined in claim 1, wherein the method comprises the following steps of: the basic structure of the ViT part in the S2 is a transducer algorithm, is based on an encoder-decoder structure, and introduces a self-attention mechanism, and the calculation formula of the self-attention mechanism is as follows:
wherein Q, K, V is matrix composed of vectors obtained by different linear transformations of input data, softmax (g) is activation function for normalization, d k Is the dimension of K.
4. The ultra-short term prediction method for wind power based on LSTM-ViT as defined in claim 3, wherein the method comprises the following steps of: the ViT part adopts a multi-head attention mechanism, and performs splicing and combination on characteristic information captured by a plurality of self-attention mechanisms to obtain a final characteristic result, wherein the calculation formula is as follows:
head i ＝Attention(QW i Q ,KW i K ,VW i V )
MultiHead(Q,K,V)＝Concat(head 1 ,…,head h )W O
wherein ,WQ 、W K、 and WV The dimensions of all three quantities are [ d model ,d K ]；W O Is [ hd ] V ,d model ]。
5. The ultra-short-term prediction method for wind power based on LSTM-ViT as defined in claim 4, wherein the method comprises the following steps of: the ViT part mainly performs matrix multiplication in the internal structure of multi-head attention, and the output of multi-head attention is subjected to nonlinear mapping by adding an activation function; each layer of the ViT encoder comprises a full-connection feedforward neural network, wherein the full-connection feedforward neural network consists of two linear transformations, and the middle is GeLU activation, and the calculation formula is as follows:
wherein W1 ,W 2 ,b 1 ,b 2 The weights and deviations of the two linear transformation layers, respectively.
6. The ultra-short-term prediction method for wind power based on LSTM-ViT as defined in claim 5, wherein the method comprises the following steps of: the ViT part transducer encoder is formed by repeatedly stacking encoder blocks for N times and superposing a plurality of transducer blocks; each transducer block consists of a multi-head self-attention module and a feedforward neural network module, which are both provided with a preposed normalization layer and are connected with the output through residual connection.
7. The ultra-short-term prediction method for wind power based on LSTM-ViT as defined in claim 6, wherein the method comprises the following steps of: the calculation formulas of the multihead self-attention module and the feedforward neural network module are as follows:
wherein , and />The outputs of the multihead self-attention module and the feedforward neural network module, respectively.
8. The ultra-short term prediction method for wind power based on LSTM-ViT as defined in claim 7, wherein the method comprises the following steps of: the LSTM-ViT wind power ultra-short-term prediction model comprises an LSTM layer and ViT layers, the time step number of the LSTM layer is 10, the batch size is 64, and the layer number is 1; the number of hidden layers in the ViT layer is 256, the number of multi-head attention mechanisms is 4, and the number of layers is 4, and the multi-head attention mechanism comprises two normalization layers, one multi-head attention mechanism and one feedforward neural network layer; the number of iterations of training was 130 and the batch size was 256.
9. The ultra-short term prediction method for wind power based on LSTM-ViT according to any one of claims 1-8, wherein the method comprises the following steps: the meteorological data includes wind speed, wind direction, temperature, air density and air pressure.
10. The ultra-short term prediction method for wind power based on LSTM-ViT as defined in claim 9, wherein the method comprises the steps of: the specific steps of carrying out normalization processing on the meteorological data and the wind power data in the S1-2 are as follows: the use of x 1 Representing power data, x 1 ' represents normalized power data, then
in the formula Cwt The installed capacity of the wind farm;
by x 2 Data representing wind speed, temperature, air pressure or air density, x 2 ' represents normalized wind speed, temperature, air pressure or air density data, then
in the formula x2max and x2min Maximum and minimum values of wind speed, temperature, air pressure or air density data respectively;
by x 3 Represents wind direction data, x 3 ' represents normalized wind direction data, then x 3 '＝sin(x 3 )。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632241.7A CN116843057A (en) | 2023-05-31 | 2023-05-31 | Wind power ultra-short-term prediction method based on LSTM-ViT |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310632241.7A CN116843057A (en) | 2023-05-31 | 2023-05-31 | Wind power ultra-short-term prediction method based on LSTM-ViT |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116843057A true CN116843057A (en) | 2023-10-03 |
Family
ID=88171553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310632241.7A Pending CN116843057A (en) | 2023-05-31 | 2023-05-31 | Wind power ultra-short-term prediction method based on LSTM-ViT |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116843057A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117313043A (en) * | 2023-10-25 | 2023-12-29 | 四川大学 | Wind power generation power prediction method |
-
2023
- 2023-05-31 CN CN202310632241.7A patent/CN116843057A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117313043A (en) * | 2023-10-25 | 2023-12-29 | 四川大学 | Wind power generation power prediction method |
| CN117313043B (en) * | 2023-10-25 | 2024-04-30 | 四川大学 | Wind power generation power prediction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492822B (en) | Air pollutant concentration time-space domain correlation prediction method | |
| CN109214575B (en) | Ultrashort-term wind power prediction method based on small-wavelength short-term memory network | |
| Xiang et al. | Ultra-short term wind power prediction applying a novel model named SATCN-LSTM | |
| Zhou et al. | Multi-energy net load forecasting for integrated local energy systems with heterogeneous prosumers | |
| CN112434848B (en) | Nonlinear weighted combination wind power prediction method based on deep belief network | |
| Li et al. | Combination of manifold learning and deep learning algorithms for mid-term electrical load forecasting | |
| US20210334658A1 (en) | Method for performing clustering on power system operation modes based on sparse autoencoder | |
| CN112861992B (en) | Wind power plant ultra-short term power prediction method based on independent sparse stacking self-encoder | |
| CN113485261B (en) | CAEs-ACNN-based soft measurement modeling method | |
| CN114693064B (en) | Building group scheme generation performance evaluation method | |
| CN116843057A (en) | Wind power ultra-short-term prediction method based on LSTM-ViT | |
| CN114676814A (en) | Wind power ultra-short-term prediction method based on SATCN-LSTM | |
| CN108520310A (en) | Wind speed forecasting method based on G-L mixed noise characteristic v- support vector regressions | |
| CN111222689A (en) | LSTM load prediction method, medium, and electronic device based on multi-scale temporal features | |
| CN116050621A (en) | Multi-head self-attention offshore wind power ultra-short-time power prediction method integrating lifting mode | |
| CN115409369A (en) | Comprehensive energy system reliability evaluation method based on mechanism and data hybrid driving | |
| CN116960978A (en) | Offshore wind power prediction method based on wind speed-power combination decomposition reconstruction | |
| CN111192158A (en) | Transformer substation daily load curve similarity matching method based on deep learning | |
| CN117439045A (en) | Multi-element load prediction method for comprehensive energy system | |
| CN115481788B (en) | Phase change energy storage system load prediction method and system | |
| Liu et al. | A hybrid deep learning model based on parallel architecture TCN-LSTM with Savitzky-Golay filter for wind power prediction | |
| CN116316629A (en) | Data-driven optimal power flow calculation method considering topology feature learning | |
| CN113762591B (en) | Short-term electric quantity prediction method and system based on GRU and multi-core SVM countermeasure learning | |
| CN116090635A (en) | Meteorological-driven new energy generation power prediction method | |
| CN115936062A (en) | Photovoltaic power day-ahead interval prediction method based on scene generation and deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |