CN110059082A - A kind of weather prediction method based on 1D-CNN and Bi-LSTM - Google Patents
A kind of weather prediction method based on 1D-CNN and Bi-LSTM Download PDFInfo
- Publication number
- CN110059082A CN110059082A CN201910307685.7A CN201910307685A CN110059082A CN 110059082 A CN110059082 A CN 110059082A CN 201910307685 A CN201910307685 A CN 201910307685A CN 110059082 A CN110059082 A CN 110059082A
- Authority
- CN
- China
- Prior art keywords
- data
- meteorological
- cnn
- lstm
- moment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/211—Schema design and management
- G06F16/212—Schema design and management with details for data modelling support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a kind of weather prediction methods based on 1D-CNN and Bi-LSTM, data and numerical forecasting product are observed based on meteorological site, data are cleaned, time warping processing is carried out to two kinds of meteorological datas, one-hot coding is carried out to website number, data normalization is carried out, data is transformed into the range intervals of 0-1, the unit limitation for removing data, is translated into nondimensional pure values；Then, the data handled well by the deep learning network training based on 1D-CNN and Bi-LSTM generate prediction model, meteorological data finally to be predicted with trained model prediction, and carry out renormalization to it, obtain final meteorological data prediction result.The precision of prediction of weather forecast can be improved using the present invention.
Description
Technical field
Present invention relates particularly to a kind of weather prediction methods based on 1D-CNN and Bi-LSTM.
Background technique
All various aspects such as daily life, social economy, the military activity of weather phenomenon and the mankind are closely related.Unexpected is sudden and violent
The natural calamities such as rain, unexpected cyclone will cause life danger and the heavy losses of property.Accurate weather forecast makes us
Unexpected heavy rain, cooling and other unexpected Changes in weather are ready, and daily life is not only facilitated, can also be
The fields such as traffic, industry, animal husbandry contribute.
Traditional weather forecast is mainly based upon the numerical weather forecast of mathematical model, and that there are precision is low for numerical forecast model
The shortcomings that.Currently, BP (backpropagation) algorithm, LSTM (shot and long term memory network) scheduling algorithm also start to be applied to weather forecast
In, but still there is a problem of that precision is lower.Therefore, a kind of weather prediction method based on deep learning need to be studied, further
Improve the precision of prediction of weather forecast.
Summary of the invention
Goal of the invention: the object of the present invention is to provide a kind of weather prediction method based on 1D-CNN and Bi-LSTM, for
The problem of conventional numeric accuracy of weather forecast deficiency is dedicated to improving the precision of prediction of weather forecast.
Technical solution: a kind of weather prediction method based on 1D-CNN and Bi-LSTM of the present invention, including following step
It is rapid:
(1) meteorological historical data is inputted, meteorological site observation data and numerical forecasting product are included；
(2) data cleansing, including suppressing exception value are carried out to meteorological historical data, default value is filled by interpolation method；
(3) by the meteorological data after the method processing cleaning of time warping, training data is formed, adaptive model knot is generated
The input format of structure；
(4) website feature is introduced, and one-hot coding is carried out to website number, N number of website is recorded by N number of feature
Website number；
(5) data normalization is carried out to data, data is transformed into the range intervals of 0-1, remove the unit limitation of data,
It is translated into nondimensional pure values；
(6) the deep learning network based on 1D-CNN and Bi-LSTM is designed, step (5) processed data are passed through in training,
Generate prediction model；
(7) meteorological data to be predicted using step (6) trained model prediction, and renormalization is carried out to it, it obtains
To final meteorological data prediction result.
Filling default value described in step (2) is realized by following formula:
Wherein, at the time of t indicates default data, m indicates there is value moment, x recently greater than t momentmIndicate the gas at m moment
As value, n table has value moment, an x in being less than t moment recentlynIndicate the meteorological value at n moment, xtFor the default of t moment to be filled
Value.
Training data described in step (3) includes that the meteorological measuring at top n moment and the numerical forecast at 0-N moment produce
Product data.
Data normalization calculation formula described in step (5) are as follows:
Wherein, x_ori indicates that, through step (4) treated data, x_min indicates the minimum value of meteorological data, x_max table
Show the maximum value of meteorological data, x indicates the output data after data normalization.
The deep learning network of 1D-CNN and Bi-LSTM described in step (6) by input layer, 1D-CNN layers, pond layer,
Bi-LSTM layers, full articulamentum and pond layer are constituted, and loss function selects mean square error loss function, and optimizer selects Adam excellent
Change device.
Data renormalization described in step (7) is realized by following formula:
X_out=x_pred × (x_max-x_min)+x_min
Wherein, x_pred indicates the data exported through step (6) network, and x_min indicates the minimum value of meteorological data, x_
Max indicates the maximum value of meteorological data, and x_out indicates the weather prognosis result after renormalization.
The utility model has the advantages that compared with prior art, beneficial effects of the present invention: the present invention proposes data cleansing, to meteorological number
According to progress time warping processing, one-hot coding station point number, the data predictions means such as data normalization, and combine 1D-
The deep learning network of CNN and Bi-LSTM, effectively increases the precision of prediction of weather forecast.
Detailed description of the invention
Fig. 1 is flow chart of the invention；
Fig. 2 is the deep learning network architecture schematic diagram that the present invention is implemented.
Specific embodiment
Present invention is further described in detail with reference to the accompanying drawing.As shown in Figure 1, the present invention the following steps are included:
1, meteorological historical data is inputted, meteorological historical data includes meteorological site observation data and numerical forecasting product.
2, data cleansing, including suppressing exception value are carried out to the meteorological historical data of input, is filled and is lacked by interpolation method
Province's value；
Each meteorological element in meteorological site observation data and numerical forecasting product has its range, deletes not in meteorology
Exceptional value within the scope of element；Default value is filled by way of linear interpolation, charging formula is as follows:
In formula, at the time of t indicates default data, m indicates there is value moment, x recently greater than t momentmIndicate the gas at m moment
As value, n table has value moment, an x in being less than t moment recentlynIndicate the meteorological value at n moment, xtFor the default of t moment to be filled
Value.
3, the data by the method processing of time warping after step 2 cleaning, generate the input lattice of adaptive model structure
Formula.
Time warping method refers to the meteorological element at prediction 0-N moment, when by the meteorological measuring and 0-N at top n moment
The numerical forecasting product data group compound training data at quarter.
4, website feature is introduced, and one-hot coding is carried out to website number, the station of N number of website is recorded by N number of feature
Point number.
One-hot coding is carried out to website number, the website number of N number of website, coding mode such as table 1 are recorded by N number of feature
It is shown.
1 coding mode of table
5, data normalization is carried out to data, data is transformed into the range intervals of 0-1, remove the unit limitation of data,
It is translated into nondimensional pure values；
Data normalization calculation formula are as follows:
In formula, x_ori indicates that, through step S4 treated data, x_min indicates the minimum value of meteorological data,
X_max indicates the maximum value of meteorological data, and x indicates the output data after data normalization.
6, design is based on the deep learning net of 1D-CNN (one-dimensional convolution) and Bi-LSTM (two-way shot and long term memory network)
Network, training arrive the processed data of step 5 by step 2, generate prediction model.
Based on the deep learning network of 1D-CNN and Bi-LSTM by input layer, 1D-CNN layers, pond layer, Bi-LSTM layers,
Full articulamentum and pond layer are constituted, and loss function selects mean square error loss function, and optimizer selects Adam optimizer.Wherein,
Deep learning network structure based on 1D-CNN and Bi-LSTM is as shown in table 2, and model is as shown in Figure 2.
The deep learning network structure of the invention of table 2
7, the meteorological data to be predicted using the trained model prediction of step 6, and renormalization is carried out to it, it obtains most
Whole meteorological data prediction result.
The calculation formula of data renormalization are as follows:
X_out=x_pred × (x_max-x_min)+x_min
In formula, x_pred indicates the data exported through step S6 network, and x_min indicates the minimum value of meteorological data, x_max
Indicate the maximum value of meteorological data, x_out indicates the weather prognosis result after renormalization.
Precision of prediction measurement index is shown below:
Wherein n is evaluation and test total sample number,For the actual observed value of i-th of sample,For the mould of i-th of sample
Type predicted value, RMSE (M) indicate the root-mean-square error of numerical weather prediction model data and truthful data, RMSE (model) table
The root-mean-square error of representation model prediction data and truthful data, total score are averaging after first calculating the scores of three prediction index
Value.
Table 3 is the result of method proposed by the present invention and the comparison of other conventional methods.TestDate be 2018 September 1
To on November 1st, 2018, totally 62 days.Wherein, it is selected 7 days in table, Testa1 is on September 24th, 2018, and Testa2 is 2018
October 15, Testb1-Testb5 are on November 1,28 days to 2018 October in 2018, and Aver_7 is 7 days average achievements,
Aver_62 is 62 days average achievements.
By comparison as it can be seen that method proposed by the present invention is better than other conventional methods, the forecast accuracy of weather forecast is more
It is high.
The implementation model of the present invention of table 3 predicts score and other methods comparing result
Claims (6)
1. a kind of weather prediction method based on 1D-CNN and Bi-LSTM, which comprises the following steps:
(1) meteorological historical data is inputted, meteorological site observation data and numerical forecasting product are included；
(2) data cleansing, including suppressing exception value are carried out to meteorological historical data, default value is filled by interpolation method；
(3) by the meteorological data after the method processing cleaning of time warping, training data is formed, adaptive model structure is generated
Input format；
(4) website feature is introduced, and one-hot coding is carried out to website number, the website of N number of website is recorded by N number of feature
Number；
(5) data normalization is carried out to data, data is transformed into the range intervals of 0-1, the unit limitation of data is removed, by it
It is converted into nondimensional pure values；
(6) the deep learning network based on 1D-CNN and Bi-LSTM is designed, step (5) processed data are passed through in training, are generated
Prediction model；
(7) meteorological data to be predicted using step (6) trained model prediction, and renormalization is carried out to it, it obtains most
Whole meteorological data prediction result.
2. a kind of weather prediction method based on 1D-CNN and Bi-LSTM according to claim 1, which is characterized in that step
Suddenly filling default value described in (2) is realized by following formula:
Wherein, at the time of t indicates default data, m indicates there is value moment, x recently greater than t momentmIndicate the meteorological value at m moment,
N table has value moment, an x in being less than t moment recentlynIndicate the meteorological value at n moment, xtFor the default value of t moment to be filled.
3. a kind of weather prediction method based on 1D-CNN and Bi-LSTM according to claim 1, which is characterized in that step
Suddenly training data described in (3) includes the meteorological measuring at top n moment and the numerical forecasting product data at 0-N moment.
4. a kind of weather prediction method based on 1D-CNN and Bi-LSTM according to claim 1, which is characterized in that step
Suddenly data normalization calculation formula described in (5) are as follows:
Wherein, x_ori indicates that, through step (4) treated data, x_min indicates the minimum value of meteorological data, and x_max indicates gas
The maximum value of image data, x indicate the output data after data normalization.
5. a kind of weather prediction method based on 1D-CNN and Bi-LSTM according to claim 1, which is characterized in that step
Suddenly the deep learning network of 1D-CNN and Bi-LSTM described in (6) by input layer, 1D-CNN layers, pond layer, Bi-LSTM layers, it is complete
Articulamentum and pond layer are constituted, and loss function selects mean square error loss function, and optimizer selects Adam optimizer.
6. a kind of weather prediction method based on 1D-CNN and Bi-LSTM according to claim 1, which is characterized in that step
Suddenly data renormalization described in (7) is realized by following formula:
X_out=x_pred × (x_max-x_min)+x_min
Wherein, x_pred indicates the data exported through step (6) network, and x_min indicates the minimum value of meteorological data, x_max table
Show the maximum value of meteorological data, x_out indicates the weather prognosis result after renormalization.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910307685.7A CN110059082A (en) | 2019-04-17 | 2019-04-17 | A kind of weather prediction method based on 1D-CNN and Bi-LSTM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910307685.7A CN110059082A (en) | 2019-04-17 | 2019-04-17 | A kind of weather prediction method based on 1D-CNN and Bi-LSTM |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110059082A true CN110059082A (en) | 2019-07-26 |
Family
ID=67317801
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910307685.7A Pending CN110059082A (en) | 2019-04-17 | 2019-04-17 | A kind of weather prediction method based on 1D-CNN and Bi-LSTM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110059082A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582560A (en) * | 2020-04-22 | 2020-08-25 | 空间信息产业发展股份有限公司 | Rice yield prediction method based on recurrent neural network |
| CN112732797A (en) * | 2021-01-26 | 2021-04-30 | 武汉理工大学 | Fuel cell automobile driving behavior analysis method, device and storage medium |
| CN113011674A (en) * | 2021-03-31 | 2021-06-22 | 电子科技大学中山学院 | Photovoltaic power generation prediction method and device, electronic equipment and storage medium |
| CN113567904A (en) * | 2021-07-02 | 2021-10-29 | 中国电力科学研究院有限公司 | Method and system suitable for metering error of capacitive mutual inductor |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622515A (en) * | 2012-02-21 | 2012-08-01 | 北京联合大学 | Weather prediction method |
| CN104978605A (en) * | 2015-06-08 | 2015-10-14 | 上海电机学院 | Large-scale wind power prediction system and method based on deep learning network |
| CN107169598A (en) * | 2017-05-05 | 2017-09-15 | 上海斐讯数据通信技术有限公司 | A kind of weather prediction method and system based on deep learning |
| CN107179566A (en) * | 2017-05-12 | 2017-09-19 | 周调彪 | The self study modification method and system of a kind of district weather forecasting |
| CN107679234A (en) * | 2017-10-24 | 2018-02-09 | 上海携程国际旅行社有限公司 | Customer service information providing method, device, electronic equipment, storage medium |
| CN107909206A (en) * | 2017-11-15 | 2018-04-13 | 电子科技大学 | A kind of PM2.5 Forecasting Methodologies based on deep structure Recognition with Recurrent Neural Network |
| CN108009674A (en) * | 2017-11-27 | 2018-05-08 | 上海师范大学 | Air PM2.5 concentration prediction methods based on CNN and LSTM fused neural networks |
| CN108399469A (en) * | 2018-01-31 | 2018-08-14 | 中国人民解放军空军研究院战场环境研究所 | A kind of weather phenomenon forecasting procedure based on deep learning and numerical weather forecast |
| CN108491877A (en) * | 2018-03-20 | 2018-09-04 | 国家气象中心 | A kind of classification convection weather probability forecast method and device |
| CN108830176A (en) * | 2018-05-25 | 2018-11-16 | 深圳市太空科技南方研究院 | A kind of sleep wakefulness detection method, device and terminal |
| CN109034152A (en) * | 2018-07-17 | 2018-12-18 | 广东工业大学 | License plate locating method and device based on LSTM-CNN built-up pattern |
| CN109637545A (en) * | 2019-01-17 | 2019-04-16 | 哈尔滨工程大学 | Based on one-dimensional convolution asymmetric double to the method for recognizing sound-groove of long memory network in short-term |
-
2019
- 2019-04-17 CN CN201910307685.7A patent/CN110059082A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622515A (en) * | 2012-02-21 | 2012-08-01 | 北京联合大学 | Weather prediction method |
| CN104978605A (en) * | 2015-06-08 | 2015-10-14 | 上海电机学院 | Large-scale wind power prediction system and method based on deep learning network |
| CN107169598A (en) * | 2017-05-05 | 2017-09-15 | 上海斐讯数据通信技术有限公司 | A kind of weather prediction method and system based on deep learning |
| CN107179566A (en) * | 2017-05-12 | 2017-09-19 | 周调彪 | The self study modification method and system of a kind of district weather forecasting |
| CN107679234A (en) * | 2017-10-24 | 2018-02-09 | 上海携程国际旅行社有限公司 | Customer service information providing method, device, electronic equipment, storage medium |
| CN107909206A (en) * | 2017-11-15 | 2018-04-13 | 电子科技大学 | A kind of PM2.5 Forecasting Methodologies based on deep structure Recognition with Recurrent Neural Network |
| CN108009674A (en) * | 2017-11-27 | 2018-05-08 | 上海师范大学 | Air PM2.5 concentration prediction methods based on CNN and LSTM fused neural networks |
| CN108399469A (en) * | 2018-01-31 | 2018-08-14 | 中国人民解放军空军研究院战场环境研究所 | A kind of weather phenomenon forecasting procedure based on deep learning and numerical weather forecast |
| CN108491877A (en) * | 2018-03-20 | 2018-09-04 | 国家气象中心 | A kind of classification convection weather probability forecast method and device |
| CN108830176A (en) * | 2018-05-25 | 2018-11-16 | 深圳市太空科技南方研究院 | A kind of sleep wakefulness detection method, device and terminal |
| CN109034152A (en) * | 2018-07-17 | 2018-12-18 | 广东工业大学 | License plate locating method and device based on LSTM-CNN built-up pattern |
| CN109637545A (en) * | 2019-01-17 | 2019-04-16 | 哈尔滨工程大学 | Based on one-dimensional convolution asymmetric double to the method for recognizing sound-groove of long memory network in short-term |
Non-Patent Citations (4)
| Title |
|---|
| CHIOU-JYE HUANG 等: ""A Deep CNN-LSTM Model for Particulate Matter (PM2.5) Forecasting in Smart Cities"", 《SENSORS 2018》 * |
| MIKHAIL STARTSEV 等: ""1D CNN with BLSTM for automated classification of fixations, saccades,and smooth pursuits"", 《HTTPS://DOI.ORG/10.3758/S13428-018-1144-2》 * |
| SHENGDONG DU 等: ""Deep Air Quality Forecasting Using Hybrid Deep Learning Framework"", 《HTTPS://ARXIV.ORG/ABS/1812.04783V1》 * |
| 倪铮 等: ""一种基于CNN和RNN深度神经网络的天气预测模型_以北京地区雷暴的6小时临近预报为例"", 《数值计算与计算机应用》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111582560A (en) * | 2020-04-22 | 2020-08-25 | 空间信息产业发展股份有限公司 | Rice yield prediction method based on recurrent neural network |
| CN111582560B (en) * | 2020-04-22 | 2023-05-23 | 空间信息产业发展股份有限公司 | Rice yield prediction method based on cyclic neural network |
| CN112732797A (en) * | 2021-01-26 | 2021-04-30 | 武汉理工大学 | Fuel cell automobile driving behavior analysis method, device and storage medium |
| CN113011674A (en) * | 2021-03-31 | 2021-06-22 | 电子科技大学中山学院 | Photovoltaic power generation prediction method and device, electronic equipment and storage medium |
| CN113567904A (en) * | 2021-07-02 | 2021-10-29 | 中国电力科学研究院有限公司 | Method and system suitable for metering error of capacitive mutual inductor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110059082A (en) | A kind of weather prediction method based on 1D-CNN and Bi-LSTM | |
| CN107480775B (en) | Pond dissolved oxygen prediction method based on data restoration | |
| CN109063903A (en) | A kind of building energy consumption prediction technique and system based on deeply study | |
| CN103023065B (en) | Wind power short-term power prediction method based on relative error entropy evaluation method | |
| CN107358294A (en) | A kind of water demand prediction method based on Elman neutral nets | |
| CN112733996B (en) | GA-PSO (genetic algorithm-particle swarm optimization) based hydrological time sequence prediction method for optimizing XGboost | |
| CN110751318A (en) | IPSO-LSTM-based ultra-short-term power load prediction method | |
| CN111079086B (en) | Multi-element joint distribution-based multiple risk assessment method for water resource system | |
| CN107705556A (en) | A kind of traffic flow forecasting method combined based on SVMs and BP neural network | |
| CN108428017A (en) | Wind power interval prediction method based on core extreme learning machine quantile estimate | |
| CN111292124A (en) | Water demand prediction method based on optimized combined neural network | |
| CN114912077B (en) | Sea wave forecasting method integrating random search and mixed decomposition error correction | |
| CN109242146A (en) | A kind of performance in layers time series predicting model based on extreme learning machine | |
| CN111915097B (en) | Water quality prediction method for optimizing LSTM neural network based on improved genetic algorithm | |
| CN114493680B (en) | Fishery resource statistical method and system based on stream stab net investigation | |
| CN115310669A (en) | Ultra-short-term wind power prediction method based on secondary decomposition and IWOA-LSSVM | |
| Shang et al. | Research on intelligent pest prediction of based on improved artificial neural network | |
| CN111105079A (en) | Internet of things water body dissolved oxygen prediction method and water quality monitoring system | |
| CN110046756B (en) | Short-term weather forecasting method based on wavelet denoising and Catboost | |
| CN116579504A (en) | Prediction method for pond mariculture water temperature based on machine learning | |
| CN110276478B (en) | Short-term wind power prediction method based on segmented ant colony algorithm optimization SVM | |
| CN117114915A (en) | Aquaculture PH value prediction method based on improved particle swarm optimization | |
| JP2021521554A (en) | Prediction method and application of Japanese flying squid resource abundance based on the Pacific decadal oscillation index | |
| CN114971005A (en) | Bay water temperature combination prediction method based on LSTM and differential regression model dynamic weighting | |
| CN113011091A (en) | Automatic-grouping multi-scale light-weight deep convolution neural network optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190726 |
|
| RJ01 | Rejection of invention patent application after publication |