CN110619432B - Feature extraction hydrological forecasting method based on deep learning - Google Patents
Feature extraction hydrological forecasting method based on deep learning Download PDFInfo
- Publication number
- CN110619432B CN110619432B CN201910874717.1A CN201910874717A CN110619432B CN 110619432 B CN110619432 B CN 110619432B CN 201910874717 A CN201910874717 A CN 201910874717A CN 110619432 B CN110619432 B CN 110619432B
- Authority
- CN
- China
- Prior art keywords
- forecasting
- flood
- model
- historical
- hydrological
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Abstract
The invention provides a feature extraction hydrological forecast method based on deep learning, belonging to the field of efficient water resource utilization and hydrological forecast, and comprising the following steps: the method comprises the steps of firstly obtaining a watershed hydrological forecast characteristic factor set by using watershed historical information, secondly combining training characteristic factor sets by using a data mining algorithm to obtain a plurality of field flood process sets with similar 'magnitude' and 'process form' under the action of different factors, then developing parameter calibration of each model and method in the traditional hydrological forecast based on a deep learning algorithm, forming a model base and a method base matched with a model and method and a parameter scheme, and finally finishing hydrological forecast calculation by combining clustering analysis. Compared with the existing method, the method effectively overcomes the defects of low forecasting precision, short effective forecasting period and the like of the traditional hydrologic forecasting method, can obviously improve the forecasting precision and prolong the forecasting period when carrying out hydrologic forecasting, has good applicability and feasibility, and provides an effective technical method for basin hydrologic forecasting.
Description
Technical Field
The invention relates to the field of efficient water resource utilization and hydrologic prediction, in particular to a deep learning-based feature extraction hydrologic prediction method.
Background
Hydrologic forecasting is an important technical support for flood and drought disaster prevention and an important means for favorable reservoir scheduling and efficient resource utilization. The hydrological forecasting method has a plurality of related models and methods, most of which can reflect some basic laws of hydrology, but because human has limited understanding on the hydrological meteorological phenomena in a drainage basin, the change of laws in the nature is complicated and intricate, the traditional models and methods are difficult to reflect objective laws comprehensively, for example, the statistical forecasting method usually faces the problem of insufficient consideration of physical significance, and the land-air coupling method often has the contradiction that the meteorological information is not matched with the spatial scale of the hydrological model.
Deep learning is used as a main branch of artificial intelligence, and is a method for continuously optimizing results through positive feedback by training with big data. With the rapid development of the internet and the internet of things, the computing capability of human beings is continuously improved, and deep learning shows excellent performance in many fields, especially in the aspects of mass data information extraction and production application.
The hydrological meteorological data is huge in quantity, wide in source and various in types, taking the Yangtze river basin as an example, the real-time observation data of dozens of units (departments) such as 15 branch centers of the Yangtze river water conservancy committee hydrological bureau, 14 provincial (directly administered city) hydrological bureaus, the China meteorological bureau, the Hubei provincial meteorological bureau, the three gorges ladder regulation center, the Jinshajiang river regulation center, all branch centralized control centers and the like are received every day, the data quantity is about 3.54 hundred million in a year, so that the exchange, sharing and information fusion of mass data are directly related to the accuracy and timeliness of flood prevention forecast scheduling of the basin, and meanwhile, a large number of information and rules which cannot be detected by conventional hydrological models and methods are hidden. However, the deep learning technology is rarely applied in hydrologic prediction so far, and information contained in and even hidden in massive data is not fully mined and utilized, so that the problems of low precision and insufficient forecast period still exist in hydrologic prediction at the present stage, and the problems of higher and higher prediction precision requirements required by epoch development and social progress are difficult to meet, and the waste of data resources is easily caused.
Therefore, based on the development of big data and artificial intelligence correlation theories and technologies, deep learning correlation theories, methods and technologies are adopted, the existing rich historical data are utilized, the physical relation among the data is expressed by using a statistical relation, the underlying surface condition of a drainage basin is described without the aid of distribution parameters as strict as a physical model, hydrologic prediction is directly realized through the statistical relation, the complexity and the complexity of hydrologic prediction work are simplified, the hydrologic prediction precision is remarkably improved, the prediction period is effectively prolonged, and the method becomes one of important research directions of the hydrologic prediction method.
Disclosure of Invention
The invention aims to provide a method for extracting hydrologic forecast based on deep learning features aiming at the defects of the prior art, so that the problems of short forecast period and low forecast precision existing in hydrologic forecast development by using a traditional hydrologic forecast model and a traditional hydrologic forecast method are solved.
In order to realize the purpose, the invention adopts the following technical scheme:
the invention provides a feature extraction hydrological forecasting method based on deep learning, which sequentially comprises the following steps of:
s1, extracting river basin historical information, and performing physical cause analysis, correlation analysis and significance test on all the historical information to obtain a river basin hydrological prediction characteristic factor set;
s2, training the hydrologic forecast characteristic factor set by using a data mining algorithm to obtain a plurality of field flood process sets with similar magnitude and process forms under the action of different factors;
s3, constructing a traditional hydrological forecasting model base and a traditional hydrological forecasting method base, developing parameter calibration of the models and the traditional hydrological forecasting methods by adopting a deep learning algorithm based on the characteristic factors and the field flood process in the step S2, obtaining a set of a plurality of groups of parameter schemes corresponding to different models and methods, and forming the model base and the method base matched with the models, the methods and the parameter schemes;
s4, extracting basic information which can be obtained by aiming at a flood process which possibly occurs in a forecast period, matching the basic information and the characteristic factor set by a cluster analysis method, further obtaining the quantity value and the process form of the flood which possibly occurs, and then selecting a corresponding model, a corresponding method and corresponding parameters from a model library and a corresponding method library to complete hydrologic forecast calculation so as to obtain forecast information;
s5, carrying out forecast effect test, judging whether the forecast precision meets the requirement, if so, ending the forecast; if not, repeating the step S4, and replacing the model, the method and the matching parameters for forecasting again until the forecasting precision meets the requirement.
Further, the step S1 includes the following steps:
s11, extracting the basin history information: classifying according to the information coverage layer and the physical cause, and dividing into an underlying surface condition and an early weather condition, wherein the underlying surface condition comprises early rainfall, runoff, early influence rainfall, soil humidity and temperature; the pre-meteorological conditions comprise 130 terms of circulation index, the 130 terms of circulation index comprising 88 terms of atmospheric circulation index, 26 terms of sea temperature index, and 16 other indices;
s12, performing physical cause analysis and correlation analysis on all historical information, wherein the calculation formula is as follows:
wherein:
in the formula: r is xy Representing a correlation coefficient between different kinds of history information x and y; s xy Representing the covariance between the heterogeneous historical information x and y; s x A standard deviation representing the history information x; s y A standard deviation representing the history information y; x is the number of i 、y i The ith individual of the historical information x and y respectively;
the average values of the historical information x and y respectively; n is the individual number of different types of historical information;
s13, carrying out significance test on the correlation analysis calculation result, wherein the calculation formula is as follows:
in the formula: SalS (I) k ) A significance value that is some type of historical information; i is k The kth individual of a certain type of historical information; i is i Any ith individual for a certain type of history information; i is a collection of some type of history information.
Further, the data mining algorithm in step S2 includes a multidimensional euclidean distance clustering method and a step function and bulldozer distance clustering method;
the multi-dimensional Euclidean distance clustering method calculates the Euclidean distances of different forecasting factors aiming at different historical flood processes so as to determine the historical flood processes with similar magnitude in different classifications, and the calculation formula is as follows:
in the formula: d (X, Y) is the Euclidean distance of a certain type of forecasting factors in the historical flood process of different times; x, Y represent historical flood courses for two different sessions, respectively; x is the number of i 、y i The ith value of a certain type of forecasting factor in the corresponding historical flood process;
the method for clustering the distance of the bulldozer calculates flood times with similar process forms in different historical flood processes, and marks the flood times as one type, wherein the calculation formula is as follows:
step function:
the distance of the bulldozer:
wherein H (t) is a step function value; f (t) is a process shape function; t is the tth moment in a flood process of a certain time; x t 、Y t Respectively are water level or flow value at the t-th moment in the flood process of two fields; EMD (X, Y) is the bulldozer distance; j. k is the j and k points in the flood process of two fields respectively; J. k is the total number of points in the flood process of two fields respectively; d jk The distance from the jth point in the first field flood process to the kth point in the second field flood process is calculated; f. of jk Is the difference in water level or flow from the jth point in the first flood run to the kth point in the second flood run.
Further, in step S3, the deep learning algorithm includes a recurrent neural network to perform parameter calibration of the traditional hydrologic prediction models, the calibration process is repeated for each type of traditional hydrologic prediction models, and the calibration results of the models are uniformly stored in a library to form a model library and a method library;
the calibration process is mainly divided into two parts of model construction and model training;
the model construction mainly comprises two processes of data normalization and model initialization;
the model training comprises two processes of forward propagation and backward propagation.
Further, the data normalization: aiming at each type of traditional hydrological forecasting model, dividing historical flood into training data and test data, and carrying out standardized processing on the training data and the test data according to a next test so as to ensure the consistency of input data;
in the formula: x i ' and X i Respectively a standard value and a real value of each vector;
and
minimum and maximum values of the input or output array, respectively; a and b are respectively normalized parameters which are positive numbers;
initializing the model: and setting calculation parameters including the number of layers for training or testing, the number of nodes, the iteration times and the termination precision.
Further, the forward propagation: aiming at a certain hydrological forecasting model to be learned, the divided training data are sequentially transmitted to an input layer, a hidden layer and an output layer for forward learning, and the deviation between a flood forecasting value and an actual value at a session is calculated according to an output result;
the reverse propagation: and based on the calculated deviation, carrying out reverse adjustment on the output layer, the hidden layer and the input layer until the calculated deviation reaches the end precision requirement or the calculated times reaches the maximum iteration times, wherein the output relevant parameters of the hydrologic prediction model are the calibration result.
Further, the forward propagation comprises:
s31, initialization parameters: random initialization U, V, W, typically 0;
s32, the calculation time t is made 0, and the calculation is performed according to the following equation:
s 1 ＝Ux 1 +Wh 0 ，h 1 ＝f(s 1 )，o 1 ＝g(Vh 1 )
s33, sequentially changing the calculation time t to t +1, and using the state of the last calculation time t-1 as the memory state to participate in the next prediction calculation, that is:
s t ＝Ux t +Wh t-1 ，h 1 ＝f(s t )，o t ＝g(Vh t )
in the formula: u, V, W are the direct weights from input layer to hidden layer, hidden layer to hidden layer, and hidden layer to output layer, respectively; s t Memory for time t; f (-) is an activation function; g (-) is typically soft max; x is a radical of a fluorine atom t Is the input at time t; h is t Is a hidden state at time t; o t Is the output at time t.
The invention has the beneficial effects that: the feature extraction hydrological forecasting method based on deep learning is realized, the defects of low forecasting precision, short effective forecasting period and the like of the traditional hydrological forecasting method are effectively overcome, the forecasting precision can be obviously improved and the forecasting period can be prolonged when hydrological forecasting is carried out, and the method has good applicability and feasibility.
1) The historical information is subjected to feature extraction by adopting various methods such as physical cause analysis, correlation analysis, significance test and the like, and is not limited to physical meaning analysis. Compared with the traditional hydrological forecasting method, the method can more quickly and accurately obtain the characteristic factors and the hidden data association relation in the historical data, can more fully, more comprehensively and more deeply utilize the existing hydrological and meteorological related data information, and has the advantages of stronger applicability, higher popularization and the like;
2) the deep learning algorithm is adopted to realize the characteristic factor training and the parameter scheme optimization, so that the historical information can be more comprehensively and more deeply utilized, and the direct matching of input and output in the hydrologic forecasting process is realized, so that a better forecasting effect is obtained, the hydrologic forecasting precision can be obviously improved, the hydrologic forecasting forecast period is effectively prolonged, and the flood and drought disaster prevention work of all watersheds in the country can be better served.
Drawings
Fig. 1 is a schematic flow chart of a feature extraction hydrological forecasting method based on deep learning according to embodiment 1 of the present invention;
fig. 2(a) is a schematic diagram of a clustering situation of historical flood provided in embodiment 1 of the present invention;
fig. 2(b) is a schematic diagram of similar results of historical flood clustering 'magnitude' provided in embodiment 1 of the present invention;
fig. 2(c) is a schematic diagram of similar results of historical flood clustering "process shape" provided in embodiment 1 of the present invention;
fig. 3 is a schematic diagram of a monthly-scale perennial runoff forecast result provided in embodiment 1 of the present invention;
fig. 4 is a schematic diagram of flood forecast results of daily-scale time periods provided in embodiment 1 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
A feature extraction hydrologic forecast method based on deep learning sequentially comprises the following steps:
s1, extracting basin historical information, and performing physical cause analysis, correlation analysis and significance test on all the historical information to obtain a basin hydrological prediction characteristic factor set;
s2, training the hydrologic forecast characteristic factor set by using a data mining algorithm to obtain a plurality of field flood process sets with similar magnitude and process forms under the action of different factors;
s3, constructing a traditional hydrological forecasting model base and a traditional hydrological forecasting method base, developing parameter calibration of the models and the traditional hydrological forecasting methods by adopting a deep learning algorithm based on the characteristic factors and the field flood process in the step S2, obtaining a set of a plurality of groups of parameter schemes corresponding to different models and methods, and forming the model base and the method base matched with the models, the methods and the parameter schemes;
s4, extracting basic information which can be obtained by aiming at a flood process which possibly occurs in a forecast period, matching the basic information and the characteristic factor set by a cluster analysis method, further obtaining the quantity value and the process form of the flood which possibly occurs, and then selecting a corresponding model, a corresponding method and corresponding parameters from a model library and a corresponding method library to complete hydrologic forecast calculation so as to obtain forecast information;
s5, carrying out forecast effect test, judging whether the forecast precision meets the requirement, if so, ending the forecast; if not, the step S4 is repeated, the model, the method and the matched parameters are replaced, and forecasting is carried out again until the forecasting precision meets the requirement.
The step S1 includes the steps of:
s11, extracting the basin history information: classifying according to the information coverage layer and the physical cause, and dividing into an underlying surface condition and an early weather condition, wherein the underlying surface condition comprises early rainfall, runoff, early influence rainfall, soil humidity and temperature; the pre-meteorological conditions comprising 130 items of circulation indices, the 130 items of circulation indices comprising 88 items of atmospheric circulation indices, 26 items of sea temperature indices, and 16 other indices;
s12, performing physical cause analysis and correlation analysis on all historical information, wherein the calculation formula is as follows:
wherein:
in the formula: r is xy Representing a correlation coefficient between the heterogeneous historical information x and y; s xy Representing the covariance between the heterogeneous historical information x and y; s. the x A standard deviation representing the history information x; s y A standard deviation representing the history information y; x is the number of i 、y i The ith individual of the historical information x and y respectively;
the average values of the historical information x and y respectively; n is the individual number of different types of historical information;
s13, carrying out significance test on the correlation analysis calculation result, wherein the calculation formula is as follows:
in the formula: SalS (I) k ) Significance value of some kind of historical information; I.C. A k The kth individual of a certain type of historical information; i is i Any ith individual for a certain type of history information; i is a collection of some type of history information.
The data mining algorithm in the step S2 includes a multidimensional euclidean distance clustering method, a step function and a bulldozer distance clustering method;
the multi-dimensional Euclidean distance clustering method calculates Euclidean distances of different forecasting factors aiming at different historical flood processes to determine historical flood processes with similar magnitude values of different classifications, and the calculation formula is as follows:
in the formula: d (X, Y) is the Euclidean distance of a certain type of forecasting factors in the historical flood process of different times; x, Y represent historical flood courses for two different sessions, respectively; x is the number of i 、y i The ith value of a certain type of forecasting factor in the corresponding historical flood process;
the method for clustering the distance of the bulldozer calculates flood times with similar process forms in different historical flood processes, and marks the flood times as one type, wherein the calculation formula is as follows:
step function:
the distance of the bulldozer:
wherein H (t) is a step function value; f (t) is a process shape function; t is the tth moment in a flood process of a certain time; x t 、Y t Respectively are water level or flow value at the t-th moment in the flood process of two fields; EMD (X, Y) is the bulldozer distance; j. k is the j and k points in the flood process of two fields respectively; J. k is the total number of points in the flood process of two fields respectively; d jk The distance from the jth point in the first field flood process to the kth point in the second field flood process is calculated; f. of jk Is the difference in water level or flow from the jth point in the first flood run to the kth point in the second flood run.
In step S3, the deep learning algorithm includes parameter calibration of the conventional hydrographic forecasting models performed by the recurrent neural network, the calibration process is repeated for each type of conventional hydrographic forecasting models, and the calibration results of the models are uniformly stored in a library to form a model library and a method library;
the calibration process is mainly divided into two parts of model construction and model training;
the model construction mainly comprises two processes of data normalization and model initialization;
the model training comprises two processes of forward propagation and backward propagation.
The data normalization comprises the following steps: aiming at each type of traditional hydrological forecasting model, dividing historical flood into training data and test data, and carrying out standardized processing on the training data and the test data according to a next test so as to ensure the consistency of input data;
in the formula: x' i And X i Respectively a standard value and a real value of each vector;
and
minimum and maximum values of the input or output array, respectively; a and b are respectively normalized parameters and are positive numbers;
initializing the model: and setting calculation parameters including the number of training or testing layers, the number of nodes, the iteration times and the termination precision.
The forward propagation: aiming at a certain hydrological forecasting model to be learned, the divided training data are sequentially transmitted to an input layer, a hidden layer and an output layer for forward learning, and the deviation between a flood forecasting value and an actual value at a session is calculated according to an output result;
the reverse propagation: and based on the calculated deviation, carrying out reverse adjustment on the output layer, the hidden layer and the input layer until the calculated deviation reaches the end precision requirement or the calculated times reaches the maximum iteration times, wherein the output relevant parameters of the hydrologic prediction model are the calibration result.
The forward propagation includes:
s31, initialization parameters: random initialization U, V, W, typically 0;
s32, the calculation time t is made 0, and the calculation is performed according to the following equation:
s 1 ＝Ux 1 +Wh 0 ，h 1 ＝f(s 1 )，o 1 ＝g(Vh 1 )
s33, sequentially changing the calculation time t to t +1, and using the state of the last calculation time t-1 as the memory state to participate in the next prediction calculation, that is:
s t ＝Ux t +Wh t-1 ，h 1 ＝f(s t )，o t ＝g(Vh t )
in the formula: u, V, W are the direct weights from input layer to hidden layer, hidden layer to hidden layer, and hidden layer to output layer, respectively; s t Memory for time t; f (-) is an activation function; g (-) is typically soft max; x is the number of t Is the input at time t; h is t Is the hidden state at time t; o t The output at time t.
Example one
The feasibility and the effectiveness of the method are verified by taking a hydrological station in the Yangtze river basin as an example. In the embodiment 1 of the invention, the month and the day are used as scales in sequence, the flow is used as a forecasting object, and the method is adopted to carry out physical cause analysis, correlation analysis and significance test on the multi-year runoff process, the historical flood processes of different occasions and corresponding basic data to obtain the hydrologic forecasting characteristic factor set of the hydrologic station. Then, training is carried out on the hydrologic forecast characteristic factor set by using a data mining algorithm described by the method of the invention, and a plurality of sets of field flood process sets with similar 'magnitude' and 'process form' under the action of different factors are obtained, which are shown in detail in figures 2(a) - (c).
Meanwhile, the method is adopted to simulate and forecast the perennial runoff process by taking months as a scale, and the forecast results of different methods are shown in figure 3. And then, on a daily scale, the method and various traditional methods are respectively adopted to carry out forecast analysis on the flood on the field, and the comparison result is shown in figure 4.
As can be seen from FIG. 3, the method of the invention can obtain better process fitting effect for the simulation prediction of the long sequence runoff process, more coverage of actual measurement points and better prediction effect for the peak value and the process than the traditional method.
As can be seen from FIG. 4, the forecasting effect of the method of the invention on the flood process of a field is superior to that of other traditional methods, the method can cover more actual measuring points, and the fitting effect on peak values and process forms is better.
Therefore, the method is high in practicability, and the problems that the traditional hydrologic prediction method is poor in prediction effect and insufficient in actually measured data coverage degree can be effectively solved.
In conclusion, the method has the advantages of high practicability, strong operability and the like, can quickly obtain the forecasting result with higher forecasting precision and longer effective forecasting period, and provides a more scientific and efficient new method for basin hydrological forecasting.
The above-mentioned embodiments only express the embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (4)
1. A feature extraction hydrological forecast method based on deep learning is characterized by sequentially comprising the following steps:
s1, extracting river basin historical information, and performing physical cause analysis, correlation analysis and significance test on all the historical information to obtain a river basin hydrological prediction characteristic factor set;
s2, training the hydrologic forecast characteristic factor set by using a data mining algorithm to obtain a plurality of field flood process sets with similar magnitude and process forms under the action of different factors;
s3, constructing a traditional hydrological forecasting model base and a traditional hydrological forecasting method base, developing parameter calibration of the models and the traditional hydrological forecasting methods by adopting a deep learning algorithm based on the characteristic factors and the field flood process in the step S2, obtaining a set of a plurality of groups of parameter schemes corresponding to different models and methods, and forming the model base and the method base matched with the models, the methods and the parameter schemes;
s4, extracting the basic information which can be obtained by the flood process which can occur in the forecast period, matching the basic information with the characteristic factor set by a cluster analysis method, further obtaining the quantity value and the process form which can occur the flood, and then selecting corresponding models, methods and matched parameters from a model base and a method base to complete hydrologic forecasting calculation so as to obtain forecasting information;
s5, carrying out forecast effect test, judging whether the forecast precision meets the requirement, if so, ending the forecast; if not, repeating the step S4, replacing the model, the method and the matched parameters for forecasting again until the forecasting precision meets the requirement;
in the step S3, the deep learning algorithm includes that a recurrent neural network carries out parameter calibration of the traditional hydrological prediction models, the calibration process is repeated for each type of traditional hydrological prediction models, and the calibration results of the models are uniformly stored in a library to form a model library and a method library;
the calibration process is mainly divided into two parts of model construction and model training;
the model construction mainly comprises two processes of data normalization and model initialization;
the model training comprises two processes of forward propagation and backward propagation;
the forward propagation: aiming at a certain hydrological forecasting model to be learned, the divided training data are sequentially transmitted to an input layer, a hidden layer and an output layer for forward learning, and the deviation between a flood forecasting value and an actual value at a session is calculated according to an output result;
the reverse propagation: based on the calculated deviation, carrying out reverse adjustment on an output layer, a hidden layer and an input layer until the calculated deviation reaches a termination precision requirement or the calculated times reaches the maximum iteration times, wherein the output relevant parameters of the hydrological prediction model are the calibration result;
the forward propagation includes:
s31, initialization parameters: random initialization U, V, W, typically 0;
s32, the calculation time t is made equal to 0, and the calculation is performed according to the following equation:
s 1 ＝Ux 1 +Wh 0 ，h 1 ＝f(s 1 )，o 1 ＝g(Vh 1 )
s33, sequentially changing the calculation time t to t +1, and using the state of the last calculation time t-1 as the memory state to participate in the next prediction calculation, that is:
s t ＝Ux t +Wh t-1 ，h 1 ＝f(s t )，o t ＝g(Vh t )
in the formula: u, V, W are the direct weight from input layer to hidden layer, the weight from hidden layer to output layer; s t Memory for time t; f (-) is an activation function; g (-) is soft max; x is the number of t Is the input at time t; h is t Is a hidden state at time t; o t The output at time t.
2. The method for extracting hydrologic forecast based on deep learning of claim 1, wherein said step S1 includes the steps of:
s11, extracting the basin history information: classifying according to the information coverage layer and the physical cause, and dividing into an underlying surface condition and an early weather condition, wherein the underlying surface condition comprises early rainfall, runoff, early influence rainfall, soil humidity and temperature; the pre-meteorological conditions comprising 130 items of circulation indices, the 130 items of circulation indices comprising 88 items of atmospheric circulation indices, 26 items of sea temperature indices, and 16 other indices;
s12, performing physical cause analysis and correlation analysis on all historical information, wherein the calculation formula is as follows:
wherein:
in the formula: r is xy Representing a correlation coefficient between different kinds of history information x and y; s xy Representing the covariance between the heterogeneous historical information x and y; s x A standard deviation representing the history information x; s y A standard deviation representing the history information y; x is the number of i 、y i The ith individual of the historical information x and y respectively;
the average values of the historical information x and y respectively; n is the individual number of different types of historical information;
s13, carrying out significance test on the correlation analysis calculation result, wherein the calculation formula is as follows:
in the formula: SalS (I) k ) Significance value of some kind of historical information; i is k The kth individual of a certain type of historical information; i is i Any ith individual for a certain type of history information; i is a collection of some type of history information.
3. The method for feature extraction hydrologic prediction based on deep learning of claim 1, wherein the data mining algorithm in step S2 comprises a multidimensional euclidean distance clustering method and a step function and bulldozer distance clustering method;
the multi-dimensional Euclidean distance clustering method calculates Euclidean distances of different forecasting factors aiming at different historical flood processes to determine historical flood processes with similar magnitude values of different classifications, and the calculation formula is as follows:
in the formula: d (X, Y) is the Euclidean distance of a certain type of forecasting factors in the historical flood process of different times; x, Y represent historical flood courses for two different sessions, respectively; x is the number of i 、y i The ith value of a certain type of forecasting factor in the corresponding historical flood process;
the method for clustering the distance of the bulldozer calculates flood times with similar process forms in different historical flood processes, and marks the flood times as one type, wherein the calculation formula is as follows:
step function:
the distance of the bulldozer:
wherein H (t) is a step function value; f (t) is a process shape function; t is the tth moment in a flood process of a certain time; x t 、Y t Respectively are water level or flow value at the t-th moment in the flood process of two fields; EMD (X, Y) is the bulldozer distance; j. k is the j and k points in the flood process of two fields respectively; J. k is the total number of points in the flood process of two fields respectively; d jk The distance from the jth point in the first field flood process to the kth point in the second field flood process is calculated; f. of jk Is the difference in water level or flow from the jth point in the first flood run to the kth point in the second flood run.
4. The method for feature extraction hydrologic forecast based on deep learning of claim 1, characterized in that said data normalization: aiming at each type of traditional hydrological forecasting model, dividing historical flood into training data and test data, and carrying out standardized processing on the training data and the test data according to a next test so as to ensure the consistency of input data;
in the formula: x i ' and X i Respectively a standard value and a real value of each vector;
and
minimum and maximum values of the input or output array, respectively; a and b are respectively normalized parameters which are positive numbers;
initializing the model: and setting calculation parameters including the number of layers for training or testing, the number of nodes, the iteration times and the termination precision.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910874717.1A CN110619432B (en) | 2019-09-17 | 2019-09-17 | Feature extraction hydrological forecasting method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910874717.1A CN110619432B (en) | 2019-09-17 | 2019-09-17 | Feature extraction hydrological forecasting method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110619432A CN110619432A (en) | 2019-12-27 |
| CN110619432B true CN110619432B (en) | 2022-08-30 |
Family
ID=68923129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910874717.1A Active CN110619432B (en) | 2019-09-17 | 2019-09-17 | Feature extraction hydrological forecasting method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110619432B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111815043B (en) * | 2020-06-30 | 2024-02-02 | 中国科学院地理科学与资源研究所 | Flood flow prediction method and system based on storm characteristics |
| CN112711917B (en) * | 2021-03-26 | 2021-07-16 | 长江水利委员会水文局 | Real-time flood forecasting intelligent method based on face recognition algorithm |
| CN113255206B (en) * | 2021-04-02 | 2023-05-12 | 河海大学 | Hydrologic prediction model parameter calibration method based on deep reinforcement learning |
| CN113435646B (en) * | 2021-06-25 | 2022-09-02 | 中国水利水电科学研究院 | Mining area ecological water demand calculation method based on annual rainfall long-term forecasting method |
| CN113570150A (en) * | 2021-08-03 | 2021-10-29 | 河海大学 | Flood forecasting method based on JSON |
| CN113723871B (en) * | 2021-11-03 | 2022-03-08 | 水利部交通运输部国家能源局南京水利科学研究院 | Multi-source information-based current situation flood consistency processing method and system |
| CN114781766B (en) * | 2022-06-22 | 2022-09-13 | 长江水利委员会长江科学院 | Hydrological information prediction method, device, equipment and storage medium for hydrological site |
| CN114971072B (en) * | 2022-06-23 | 2023-06-13 | 陕西省水文水资源勘测中心 | Hydrologic forecasting system based on multi-factor similarity analysis |
| CN116091801B (en) * | 2023-03-07 | 2023-06-16 | 河海大学 | Rainfall image similarity searching method based on deep learning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101864750A (en) * | 2010-06-29 | 2010-10-20 | 西安理工大学 | Multi-model meta-synthesis flood forecasting system and forecasting method thereof |
| CN106650767A (en) * | 2016-09-20 | 2017-05-10 | 河海大学 | Flood forecasting method based on cluster analysis and real time correction |
| CN106845771A (en) * | 2016-12-16 | 2017-06-13 | 中国水利水电科学研究院 | A kind of Flood Forecasting Method based on previous rainfall amount preferred parameter |
| CN106875047A (en) * | 2017-01-23 | 2017-06-20 | 国网湖南省电力公司 | Reservoir watershed Runoff Forecast method and system |
| CN108021773A (en) * | 2017-12-27 | 2018-05-11 | 大连理工大学 | The more play flood parameters rating methods of hydrological distribution model based on DSS data base read-writes |
| CN108764515A (en) * | 2018-04-04 | 2018-11-06 | 河海大学 | A kind of reservoir operation Application of risk decision method of Coupled Numerical meteorological model DATA PROCESSING IN ENSEMBLE PREDICTION SYSTEM |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102628876B (en) * | 2012-02-13 | 2013-07-31 | 甘肃省电力公司风电技术中心 | Ultra-short term prediction method comprising real-time upstream and downstream effect monitoring |
-
2019
- 2019-09-17 CN CN201910874717.1A patent/CN110619432B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101864750A (en) * | 2010-06-29 | 2010-10-20 | 西安理工大学 | Multi-model meta-synthesis flood forecasting system and forecasting method thereof |
| CN106650767A (en) * | 2016-09-20 | 2017-05-10 | 河海大学 | Flood forecasting method based on cluster analysis and real time correction |
| CN106845771A (en) * | 2016-12-16 | 2017-06-13 | 中国水利水电科学研究院 | A kind of Flood Forecasting Method based on previous rainfall amount preferred parameter |
| CN106875047A (en) * | 2017-01-23 | 2017-06-20 | 国网湖南省电力公司 | Reservoir watershed Runoff Forecast method and system |
| CN108021773A (en) * | 2017-12-27 | 2018-05-11 | 大连理工大学 | The more play flood parameters rating methods of hydrological distribution model based on DSS data base read-writes |
| CN108764515A (en) * | 2018-04-04 | 2018-11-06 | 河海大学 | A kind of reservoir operation Application of risk decision method of Coupled Numerical meteorological model DATA PROCESSING IN ENSEMBLE PREDICTION SYSTEM |
Non-Patent Citations (3)
| Title |
|---|
| A Method of Rainfall Runoff Forecasting Based on Deep Convolution Neural Networks;Xiaoli Li，Zhenlong Du;《2018 Sixth International Conference on Advanced Cloud and Big Data》;20180831;全文 * |
| 基于模糊聚类和BP神经网络的流域洪水分类预报研究;任明磊等;《大连理工大学学报》;20090115(第01期);全文 * |
| 浅析水库长期水文预报系统的多预报思路;李永强;《珠江水运》;20150930;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110619432A (en) | 2019-12-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619432B (en) | Feature extraction hydrological forecasting method based on deep learning | |
| CN113379109B (en) | Runoff forecasting method based on prediction model self-adaption | |
| CN108304668B (en) | Flood prediction method combining hydrologic process data and historical prior data | |
| CN101480143B (en) | Method for predicating single yield of crops in irrigated area | |
| Kumar et al. | Artificial neural network model for rainfall-runoff-A case study | |
| CN107292098A (en) | Medium-and Long-Term Runoff Forecasting method based on early stage meteorological factor and data mining technology | |
| CN107274030B (en) | Runoff Forecast method and system based on hydrology variable year border and monthly variation characteristic | |
| CN109635245A (en) | A kind of robust width learning system | |
| CN108021773B (en) | DSS database-based distributed hydrological model multi-field secondary flood parameter calibration method | |
| Remesan et al. | Effect of data time interval on real-time flood forecasting | |
| CN113139329B (en) | Xinanjiang model parameter calibration method based on hydrological similarity and artificial neural network | |
| CN112182063A (en) | Method for constructing hydrological forecasting model based on space-time characteristics | |
| CN114444378A (en) | Short-term power prediction method for regional wind power cluster | |
| Chen et al. | CRML: A convolution regression model with machine learning for hydrology forecasting | |
| CN112396152A (en) | Flood forecasting method based on CS-LSTM | |
| CN115759445A (en) | Machine learning and cloud model-based classified flood random forecasting method | |
| CN116432828A (en) | Intelligent prediction method for runoff of data-missing river basin | |
| CN111914488B (en) | Data area hydrologic parameter calibration method based on antagonistic neural network | |
| Wang et al. | Two-dimension monthly river flow simulation using hierarchical network-copula conditional models | |
| CN110969312A (en) | Short-term runoff prediction coupling method based on variational modal decomposition and extreme learning machine | |
| WO2022032873A1 (en) | Adversarial neural network-based hydrological parameter calibration method for data-lacking region | |
| CN112036604B (en) | Medium runoff forecasting method considering multiple time sequence process factors | |
| CN116796799A (en) | Method for creating small-river basin flood rainfall threshold model in area without hydrologic data | |
| CN113159224A (en) | Runoff forecasting model construction method and device, electronic equipment and medium | |
| Parviz et al. | A comparison of the efficiency of parameter estimation methods in the context of streamflow forecasting. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |