CN111814407B - Flood forecasting method based on big data and deep learning - Google Patents
Flood forecasting method based on big data and deep learning Download PDFInfo
- Publication number
- CN111814407B CN111814407B CN202010735098.0A CN202010735098A CN111814407B CN 111814407 B CN111814407 B CN 111814407B CN 202010735098 A CN202010735098 A CN 202010735098A CN 111814407 B CN111814407 B CN 111814407B
- Authority
- CN
- China
- Prior art keywords
- data
- flood
- water level
- value
- measuring device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Abstract
The invention relates to a flood forecasting method based on big data and deep learning, which comprises the following steps of; collecting and arranging data: collecting and recording historical recorded local duration rainfall data, collecting and recording historical recorded local river flow data, and sorting the duration rainfall data and the river flow data, wherein the data screening step comprises the steps of: and screening the maximum value and the minimum value of the number of the watershed in each time period, comparing the time points of the occurrence of the historical flood, and determining a trigger value close to the flood and a minimum water level value. The method solves the problems of complex physical modeling and poor portability of the traditional flood forecasting method. The technology extracts the rainfall and water quantity (water level) change conditions during the occurrence of historical flood, and simultaneously, the LSTM neural network is used for forecasting the flood water quantity and water level information in combination with the time sequence characteristics of the rainfall change during the period. The result shows that the technology has higher precision and accuracy compared with the traditional flood forecasting system.
Description
Technical Field
The invention relates to the technical field of flood forecasting, in particular to a flood forecasting method based on big data and deep learning.
Background
Flood forecast provides basis for flood control emergency and water resource scheduling by predicting short, medium and long-term flood occurrence and change trend, provides water condition guarantee for normal life of enterprises and institutions along the river and life and property safety of residents, and can avoid a lot of unnecessary losses by immediately withdrawing when flood is sudden.
The traditional flood forecasting method is an empirical forecasting method based on a certain physical model, has high professional requirements, and has poor portability of the model. With the improvement of data acquisition technology, the accuracy of the collected data is improved, and the data volume is increased, so that the flood forecast by adopting big data and deep learning technology is possible.
Disclosure of Invention
(one) solving the technical problems
Aiming at the defects of the prior art, the invention provides a flood forecasting method based on big data and deep learning, which solves the problems of complex physical modeling and poor portability of the traditional flood forecasting method.
(II) technical scheme
In order to achieve the above purpose, the present invention provides the following technical solutions: a flood forecasting method based on big data and deep learning, comprising;
collecting and arranging data: collecting and recording historical recorded local duration rainfall data, collecting and recording historical recorded local river flow data, and sorting the duration rainfall data and the river flow data;
and (3) data screening: screening the maximum value and the minimum value of the quantity of the watershed in each time period, comparing the time points of the occurrence of the historical flood, and determining a trigger value close to the flood and a minimum water level value;
the step of establishing a sample: establishing the size of a sample, converting data into the form of the number of samples and the sequence length;
learning: in deep learning, setting the number of hidden units, constructing an LSTMS, optimizing a grid, obtaining an optimal LSTMS model when the data loss is small, and continuing optimizing the grid until the optimal LSTMS model is obtained and predicting the water level of the water quantity when the data loss is large;
and the test set compares the measured result data with the historical data to obtain the numerical values of the water quantity and the water level, so that the occurrence probability of the flood is predicted.
Preferably, the test set comprises a water level measuring device for water level measurement and a precipitation measuring device for precipitation measurement, and the test set performs verification comparison on data collected by the precipitation measuring device and the water level measuring device, wherein the compared data is used as a verification node every fifteen minutes according to a time node, namely, the data collected by the precipitation measuring device and the water level measuring device are fed back every fifteen minutes.
Preferably, in the learning step, when the data loss is serious, if the flood exists in the missing data year, the peak value of the flood is recorded, if the flood does not exist in the missing data, the data is recorded according to the overall average value or compared with the regional data close to the region, and otherwise, if the hand-recorded data exists in the hydrological station, each data is recorded, and a data model of the year is generated.
Preferably, the data of the test set includes water level data and precipitation data, when the water level data and the precipitation data are put into an optimal LSTMS model, the data in the same season in the optimal LSTMS model are compared, and the data in the same season for many years are compared, so that an optimal comparison scheme is obtained through analysis, the situation of flood in the time when the data exist is analyzed, and when the critical value is reached, the risk and probability of the flood in the area are prompted to rise.
(III) beneficial effects
The invention provides a flood forecasting method based on big data and deep learning. The beneficial effects are as follows:
(1) The invention realizes the flood forecasting method based on big data, establishes time sequence analysis, and performs flood forecasting through a given learning sample, thereby avoiding complex hydrologic process analysis.
(2) The technology extracts the rainfall and water (water level) change conditions during the occurrence of historical flood, and simultaneously, the LSTM neural network is used for forecasting the flood water and water level information in combination with the time sequence characteristics of the rainfall change during the period. The result shows that the technology has higher precision and accuracy compared with the traditional flood forecasting system.
Drawings
Fig. 1 is a flow chart of flood forecast according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, the present invention provides a technical solution: a flood forecasting method based on big data and deep learning, comprising;
collecting and arranging data: collecting and recording historical recorded local duration rainfall data, collecting and recording historical recorded local river flow data, and sorting the duration rainfall data and the river flow data, wherein the sorted data comprise data recorded by hydrologic stations, and the sorted data also comprise historical flood occurrence conditions and recorded flood data when flood occurs;
and (3) data screening: screening the maximum value and the minimum value of the number of the watershed in each time period, comparing the time points of the occurrence of the historical flood, determining a trigger value close to the flood and a minimum water level value, and carrying out flood data values of each year;
the step of establishing a sample: establishing the size of a sample, converting the data into the number of samples, and establishing a model by the data of unit time, minutes, hours, half days, one day, one week and one month in the form of a sequence length;
learning: in deep learning, setting the number of hidden units, constructing an LSTMS, optimizing a grid, obtaining an optimal LSTMS model when the data loss is small, and continuing optimizing the grid until the optimal LSTMS model is obtained and predicting the water level of the water quantity when the data loss is large;
and the test set compares the measured result data with the historical data to obtain the numerical values of the water quantity and the water level, so that the occurrence probability of the flood is predicted.
Specifically, the test set comprises a water level measuring device for water level measurement and a precipitation measuring device for precipitation measurement, and the test set verifies and compares data collected by the precipitation measuring device and the water level measuring device, wherein the compared data is used as a verification node every fifteen minutes according to a time node, namely, the data collected by the precipitation measuring device and the water level measuring device are fed back every fifteen minutes.
Specifically, in the learning step, when the data loss is serious, if the flood exists in the missing data year, the peak value of the flood is recorded, if the flood does not exist in the missing data, the data is recorded according to the overall average value or compared with the regional data close to the region, and in addition, if the hand-recorded data exists in the hydrological station, each data is recorded, and a data model of the year is generated.
Specifically, when the water level data and the precipitation data are put into the optimal LSTMS model, comparing the water level data and the precipitation data according to the data in the same season in the optimal LSTMS model and the data in the same season for many years, analyzing to obtain an optimal comparison scheme, analyzing the flood condition in the time of the data, and prompting the risk and probability rising of the flood in the area when the critical value is reached.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a reference structure" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (3)
1. The flood forecasting method based on big data and deep learning is characterized by comprising the following steps of;
collecting and arranging data: collecting and recording historical recorded local duration rainfall data, collecting and recording historical recorded local river flow data, and sorting the duration rainfall data and the river flow data;
and (3) data screening: screening the maximum value and the minimum value of the local river flow in each time period, comparing the time points of the occurrence of the historical flood, and determining the latest value and the lowest water level value of a flood trigger value;
the step of establishing a sample: establishing the size of a sample, converting data into the form of the number of samples and the sequence length;
learning: in deep learning, setting the number of hidden units, constructing an LSTMS, optimizing a grid, obtaining an optimal LSTMS model when the data loss is small, and continuing optimizing the grid until the optimal LSTMS model is obtained and the water level of the water is predicted when the data loss is larger than a preset value;
the method comprises the steps of comparing measured result data with historical data to obtain water quantity and water level values, so that the occurrence probability of flood is predicted, wherein the data of the test set comprise water level data and precipitation data, when the water level data and the precipitation data are placed in an optimal LSTMS model, comparing the data in the same season in the optimal LSTMS model and data of many years in the same season, analyzing to obtain an optimal comparison scheme, and analyzing the flood condition in the time of the data, and when a critical value is reached, prompting the increase of the risk and probability of local flood.
2. The flood forecast method based on big data and deep learning according to claim 1, characterized in that: the test set comprises a water level measuring device for water level measurement and a precipitation measuring device for precipitation measurement, and the test set verifies and compares data collected by the precipitation measuring device and the water level measuring device, wherein the compared data is used as a verification node every fifteen minutes according to a time node, namely, the data collected by the precipitation measuring device and the water level measuring device are fed back every fifteen minutes.
3. The flood forecast method based on big data and deep learning according to claim 1, characterized in that: in the learning step, when the local data is seriously lost, if flood exists in the lost data year, the peak value of the flood is recorded, if the lost data is not flood, the data is recorded according to the overall average value or is compared with the local area data, and in addition, if the hand-recorded data exists in the hydrological station, each data is recorded, and a data model of the year is generated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010735098.0A CN111814407B (en) | 2020-07-28 | 2020-07-28 | Flood forecasting method based on big data and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010735098.0A CN111814407B (en) | 2020-07-28 | 2020-07-28 | Flood forecasting method based on big data and deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111814407A CN111814407A (en) | 2020-10-23 |
| CN111814407B true CN111814407B (en) | 2023-10-13 |
Family
ID=72862756
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010735098.0A Active CN111814407B (en) | 2020-07-28 | 2020-07-28 | Flood forecasting method based on big data and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111814407B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712194A (en) * | 2020-12-16 | 2021-04-27 | 广西电网有限责任公司梧州供电局 | Electric quantity prediction method and device for power consumption cost intelligent optimization analysis |
| CN113378484B (en) * | 2021-07-12 | 2021-12-21 | 中国水利水电科学研究院 | Plain river network area flood process forecasting method based on machine learning |
| CN115014299B (en) * | 2022-08-10 | 2022-11-18 | 山脉科技股份有限公司 | Flood peak early warning method based on Internet of things and big data |
| CN115293469B (en) * | 2022-10-10 | 2022-12-27 | 山脉科技股份有限公司 | Urban flood control and drainage risk prediction method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408900A (en) * | 2014-11-10 | 2015-03-11 | 柳州师范高等专科学校 | Dynamic optimization based neural network flood warning device and method |
| CN108388957A (en) * | 2018-01-25 | 2018-08-10 | 河海大学 | A kind of middle and small river Flood Forecasting Method and its forecast system based on multiple features fusion technology |
| CN109341814A (en) * | 2018-12-10 | 2019-02-15 | 安徽沃特水务科技有限公司 | A kind of all the period of time water level collection system and method |
| JP2019045290A (en) * | 2017-09-01 | 2019-03-22 | 東急建設株式会社 | River water level prediction system |
| CN109583565A (en) * | 2018-11-07 | 2019-04-05 | 河海大学 | Forecasting Flood method based on the long memory network in short-term of attention model |
| CN110471950A (en) * | 2019-07-19 | 2019-11-19 | 河海大学 | A kind of middle and small river Real-time Flood Forecasting model of mind forecasting procedure |
-
2020
- 2020-07-28 CN CN202010735098.0A patent/CN111814407B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408900A (en) * | 2014-11-10 | 2015-03-11 | 柳州师范高等专科学校 | Dynamic optimization based neural network flood warning device and method |
| JP2019045290A (en) * | 2017-09-01 | 2019-03-22 | 東急建設株式会社 | River water level prediction system |
| CN108388957A (en) * | 2018-01-25 | 2018-08-10 | 河海大学 | A kind of middle and small river Flood Forecasting Method and its forecast system based on multiple features fusion technology |
| CN109583565A (en) * | 2018-11-07 | 2019-04-05 | 河海大学 | Forecasting Flood method based on the long memory network in short-term of attention model |
| CN109341814A (en) * | 2018-12-10 | 2019-02-15 | 安徽沃特水务科技有限公司 | A kind of all the period of time water level collection system and method |
| CN110471950A (en) * | 2019-07-19 | 2019-11-19 | 河海大学 | A kind of middle and small river Real-time Flood Forecasting model of mind forecasting procedure |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111814407A (en) | 2020-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111814407B (en) | Flood forecasting method based on big data and deep learning | |
| CN109615226B (en) | Operation index abnormity monitoring method | |
| CN110110912B (en) | Photovoltaic power multi-model interval prediction method | |
| CN103678766B (en) | A kind of abnormal Electricity customers detection method based on PSO algorithm | |
| CN107992961A (en) | A kind of adaptive basin Medium-and Long-Term Runoff Forecasting model framework method | |
| CN109740195B (en) | Extreme value typhoon wind speed probability distribution model based on meteorological station observation data and evaluation method for designing typhoon wind speed | |
| CN108734342B (en) | Method for forecasting regional weather by using regional power grid overhead line heat setting probability | |
| CN113496311A (en) | Photovoltaic power station generated power prediction method and system | |
| CN112330065A (en) | Runoff forecasting method based on basic flow segmentation and artificial neural network model | |
| CN110598352B (en) | Drainage basin water supply forecasting method | |
| Cui et al. | Algorithm for identifying wind power ramp events via novel improved dynamic swinging door | |
| CN114006369A (en) | Regional wind and light station power joint prediction method and device, electronic equipment and storage medium | |
| CN113256326A (en) | Method for realizing prediction of commodity extra-large screen point position pedestrian volume based on deep learning | |
| CN108960522A (en) | A kind of photovoltaic power generation quantity prediction analysis method | |
| CN114357670A (en) | Power distribution network power consumption data abnormity early warning method based on BLS and self-encoder | |
| CN112287299B (en) | Quantitative attribution method, device and system for river health change | |
| CN116756505A (en) | Photovoltaic equipment intelligent management system and method based on big data | |
| CN109829572B (en) | Photovoltaic power generation power prediction method under thunder and lightning weather | |
| CN115733258A (en) | Control method of all-indoor intelligent substation system based on Internet of things technology | |
| CN108876060A (en) | A kind of sample collection wind power output probability forecasting method based on big data | |
| CN114462802A (en) | Non-stationary drought index construction method based on climate index and improved reservoir index | |
| CN110852516B (en) | Traffic flow detection equipment data quality discriminating method based on big data information entropy | |
| CN114819260A (en) | Dynamic generation method of hydrologic time series prediction model | |
| CN114529071A (en) | Method for predicting power consumption of transformer area | |
| CN113589404A (en) | Method for predicting runoff of storm of field |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |