CN113496104A - Rainfall forecast correction method and system based on deep learning - Google Patents

Rainfall forecast correction method and system based on deep learning Download PDF

Info

Publication number
CN113496104A
CN113496104A CN202110804467.1A CN202110804467A CN113496104A CN 113496104 A CN113496104 A CN 113496104A CN 202110804467 A CN202110804467 A CN 202110804467A CN 113496104 A CN113496104 A CN 113496104A
Authority
CN
China
Prior art keywords
forecast
precipitation
rainfall
data
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110804467.1A
Other languages
Chinese (zh)
Other versions
CN113496104B (en
Inventor
杨春蕾
袁慧玲
谢梦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Institute Of Technical Physics
Original Assignee
Suzhou Institute Of Technical Physics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Institute Of Technical Physics filed Critical Suzhou Institute Of Technical Physics
Priority to CN202110804467.1A priority Critical patent/CN113496104B/en
Publication of CN113496104A publication Critical patent/CN113496104A/en
Application granted granted Critical
Publication of CN113496104B publication Critical patent/CN113496104B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01W—METEOROLOGY
    • G01W1/00—Meteorology
    • G01W1/10—Devices for predicting weather conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2111/00—Details relating to CAD techniques
    • G06F2111/08—Probabilistic or stochastic CAD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2113/00—Details relating to the application field
    • G06F2113/08—Fluids
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Environmental & Geological Engineering (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Atmospheric Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Ecology (AREA)
  • Environmental Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computer Hardware Design (AREA)
  • Geometry (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to a rainfall forecast correction method and system based on deep learning, wherein the rainfall forecast correction method comprises the following steps: preprocessing forecast data of observed rainfall and mode output rainfall by using a space-time interpolation method so as to enable the forecast data to have the same space-time resolution; inputting satellite data which can respectively represent water vapor characteristics and vertical motion characteristics into a circulating neural network to extract time sequence characteristics of the satellite data; inputting the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution into a convolutional neural network, defining weight factors of loss functions in the convolutional neural network, and correcting the forecast data after model training to obtain a corrected precipitation result. The invention selects a proper deep learning model, improves the precipitation space distribution by increasing the context information, improving the weight and the like, and corrects a large amount of precipitation by adjusting the loss function and the weight factor, thereby solving the general underestimation large-value precipitation problem.

Description

Rainfall forecast correction method and system based on deep learning
Technical Field
The embodiment of the invention relates to the field of data encryption, in particular to a rainfall forecast correcting method and system based on deep learning.
Background
Precipitation is one of the major drivers of global hydrologic cycles and is a key component in regulating climate systems. Among a plurality of natural disasters, flood disasters caused by extreme rainfall are the most serious natural disasters, and are one of the main factors for restricting the sustainable development of the society and the economy at present. The quantitative precipitation estimation with high space-time resolution is important for extreme weather early warning, the uncertainty of precipitation is evaluated and reduced, and the method becomes an important research direction for the early warning of extreme meteorological disasters and extreme hydrological disasters.
Quantitative precipitation estimation remains a challenging problem due to the large spatial and temporal variation of precipitation, where deviations in the forecasted precipitation are important causes of impact on the results of the hydrologic forecast. The current common rainfall ensemble prediction correction method is based on a mathematical statistic method, and comprises frequency matching, optimal percentile, probability matching average, multi-mode dynamic weight integration, Analog historical similarity method, logistic regression method, Bayesian model average method, non-homogeneous regression and the like. However, in the existing precipitation correction method, due to the limited number of samples of extreme precipitation and the structural problem of a precipitation correction statistical model, the phenomenon of large-value general underestimation precipitation exists, and the application of ensemble prediction precipitation in hydrological simulation is greatly influenced.
Disclosure of Invention
The embodiment of the invention provides a rainfall forecast correction method and system based on deep learning, and aims to solve the technical problems.
In order to solve the above technical problem, the embodiment of the present invention adopts a technical solution that: the method for correcting the rainfall forecast based on the deep learning comprises the following steps:
preprocessing forecast data of observed rainfall and mode output rainfall by using a space-time interpolation method so as to enable the forecast data to have the same space-time resolution;
inputting satellite data which can respectively represent water vapor characteristics and vertical motion characteristics into a circulating neural network to extract time sequence characteristics of the satellite data;
inputting the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution into a convolutional neural network, defining weight factors of loss functions in the convolutional neural network, and correcting the forecast data after model training to obtain a corrected precipitation result.
In a preferred embodiment of the present invention, the forecast data includes a precipitation error distribution characteristic and an influence characteristic on the runoff simulation.
In a preferred embodiment of the present invention, the loss function L is a weight factor defining a loss function in the convolutional neural networkestimationRelative error of plane rainfall LRE+ spatial correlation system LCCWherein, in the step (A),
Figure BDA0003165884770000021
Figure BDA0003165884770000022
wherein n is the total number of events, OiAnd SiI for the observed precipitation and the precipitation forecast to be tested,
Figure BDA0003165884770000023
and
Figure BDA0003165884770000024
the average values of the observed precipitation and the precipitation forecast are respectively.
As a preferable aspect of the present invention, when the convolutional neural network corrects the forecast data, precipitation spatial distribution information is also obtained.
As a preferred scheme of the present invention, a vortex pooling module is disposed in the convolutional neural network, and when the convolutional neural network obtains the precipitation spatial distribution information, context information is obtained through the vortex pooling module.
As a preferred scheme of the present invention, before the convolutional neural network corrects the forecast data, model training is performed using the observed precipitation as a label, and model training parameters are retained.
As a preferable scheme of the present invention, the data inspection and evaluation of the correction precipitation result specifically includes:
carrying out deterministic test and evaluation on the forecasting capacity of correcting the precipitation result through a preset scoring strategy;
and carrying out probabilistic test and evaluation on the forecasting capacity for correcting the precipitation result through probability operation.
As a preferred scheme of the present invention, the predetermined scoring strategy includes calculating and evaluating a fair forecasting rate ETS, a Frequency deviation Frequency Bias, a hit rate POD, and a false alarm rate FAR, respectively, and the calculation formula is specifically as follows:
Figure BDA0003165884770000031
Figure BDA0003165884770000032
Figure BDA0003165884770000033
Figure BDA0003165884770000034
h represents the number of grid points correctly forecasted, M represents the number of missed report grid points, F represents the number of empty report grid points, and C represents the number of grid points correctly forecasted without precipitation events.
As a preferable aspect of the present invention, the probabilistic test and evaluation of the forecast capability for correcting precipitation results through probability operation specifically includes:
after comparing the set dispersion of the set members in the corrected precipitation result with the mean root-mean-square error of the set, analyzing the error relation of the set dispersion to check the dispersion state of the set members,
will f isi(n) is reported as the predicted value of the nth set member of the ith sample, wherein i is 1,2,3, …, M; n-1, 2,3, …, N; m is the total number of samples, N is the set membership, andithe calculation formula is expressed as the observation of the ith sample, and is specifically as follows:
the ensemble average employed is that of,
Figure BDA0003165884770000035
the root mean square error is as follows,
Figure BDA0003165884770000036
the set of dispersion degrees is that,
Figure BDA0003165884770000037
drawing a working characteristic curve of the testee according to the calculation results of the hit rate and the false alarm rate to judge the forecasting skill;
performing a BSs score calculation based on the BS score result to compare the forecast probability and the actual occurrence probability, wherein,
will PiRecord as the forecast probability of sample i, let O beiAnd (3) recording as the actual occurrence probability of the sample i, wherein the calculation formula is as follows:
the BS score is such that,
Figure BDA0003165884770000041
Figure BDA0003165884770000042
the BSS score is such that,
Figure BDA0003165884770000043
in order to solve the above technical problem, an embodiment of the present invention further provides a rainfall forecast correcting system based on deep learning, including:
the preprocessing unit is used for preprocessing forecast data of observed rainfall and mode output rainfall so as to enable the forecast data to have the same space-time resolution;
the circulating neural network is used for receiving satellite data which can respectively represent water vapor characteristics and vertical motion characteristics so as to extract time sequence characteristics of the satellite data;
the convolutional neural network is used for receiving the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution, and defining the weight factor of the loss function so as to correct the forecast data after model training to obtain a corrected precipitation result; and the number of the first and second groups,
and the inspection and evaluation unit is used for performing data inspection and evaluation on the corrected precipitation result.
In a preferred embodiment of the present invention, the forecast data includes a precipitation error distribution characteristic and an influence characteristic on the runoff simulation.
In order to solve the above technical problem, an embodiment of the present invention further provides a computer device, including a memory and a processor, where the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the rainfall forecast correcting method.
In order to solve the above technical problem, an embodiment of the present invention further provides a storage medium storing computer readable instructions, which when executed by one or more processors, cause the one or more processors to execute the steps of the precipitation forecast correcting method.
In summary, the embodiments of the present invention have the following beneficial effects:
the embodiment of the invention provides a rainfall forecast correcting method and system based on deep learning, and the method and system are combined with a physical mechanism, utilize a recurrent neural network model to extract the time sequence characteristics of water vapor and vertical movement in rainfall forming conditions, and input elevation data into a convolutional neural network model together based on the influence of terrain on rainfall, wherein the depth of the method is combined with a weather theory and an artificial intelligence model, firstly utilizes the recurrent neural network to extract physical characteristics, and then inputs the physical characteristics into a model structure of the convolutional neural network, so that the precision of rainstorm magnitude rainfall and the spatial distribution of the rainfall are improved in a targeted manner, and the performance of rainfall estimation is effectively improved; meanwhile, based on prior information such as influence of rainfall error distribution on runoff simulation, an appropriate deep learning model is selected for improving the runoff simulation, the problem of insufficient extraction of extreme rainfall forecast information is solved by emphatically improving rainfall spatial distribution through increasing context information, improving weight and the like, a large amount of levels of rainfall is emphatically corrected by adjusting a loss function and a weight factor, and the problem of general underestimation of large-value rainfall is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a basic flow of a method for correcting precipitation forecast according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data processing flow of a recurrent neural network and a convolutional neural network according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a recurrent neural network structure according to an embodiment of the present invention.
FIG. 4 is a schematic view of a vortex pooling module in an embodiment of the present invention.
FIG. 5 is a timing diagram of forecasted precipitation and observed precipitation before and after deep learning correction in an embodiment of the invention.
Fig. 6 is a schematic structural diagram of a precipitation forecast correction system according to an embodiment of the present invention.
Fig. 7 is a block diagram of a basic structure of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
In some of the flows described in the present specification and claims and in the above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein, with the order of the operations being indicated as 101, 102, etc. merely to distinguish between the various operations, and the order of the operations by themselves does not represent any order of performance. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic diagram of a basic flow of the method for correcting precipitation forecast according to the present embodiment, and fig. 2 is a schematic diagram of a data processing flow of a recurrent neural network and a convolutional neural network in the method for correcting precipitation forecast according to the present embodiment.
A rainfall forecast correction method based on deep learning comprises the following steps:
s100, preprocessing observed precipitation and mode output precipitation forecast data by using a space-time interpolation method so that the forecast data have the same space-time resolution, wherein the observed precipitation generally refers to the precipitation observed by a ground station, the mode output precipitation generally refers to the precipitation processed by a precipitation forecast processing system, the forecast data generally also comprises precipitation error distribution characteristics and influence characteristics on runoff simulation, and the preprocessing process can be performed in the existing precipitation forecast processing system.
Because the spatial-temporal resolution of the mode output precipitation (1 degree, 6h) and the observation precipitation (0.25 degree, 24h) are not matched, in order to train, test and evaluate the deep learning model, the same spatial-temporal resolution needs to be obtained through a pretreatment process, and the specific method of the pretreatment comprises the following steps: the mode output precipitation is spatially enabled to have the same resolution (0.25 degrees) as the observed precipitation by using a bilinear interpolation method, and the daily cumulative precipitation is obtained by adding up every 6 hours of precipitation of the mode output precipitation in time.
And S200, inputting satellite data which can respectively represent water vapor characteristics and vertical motion characteristics into a circulating neural network to extract time sequence characteristics of the satellite data.
Among them, a Recurrent Neural Network (RNN) is a deep learning model that models with sequence data as an input. Unlike the fully connected neural network layer, the recurrent neural network has a feedback connection that allows past information to affect the current output, so the output at the current time in the recurrent neural network is the accumulation of its current inputs and the output at the previous moment. With this structure, the recurrent neural network has the ability to memorize past information, which takes a certain advantage in learning the nonlinear characteristics of sequences by virtue of the characteristics of memorability, parameter sharing, and graph completion (training completion).
Specifically referring to fig. 3, fig. 3 is a schematic structural diagram of the recurrent neural network structure in this embodiment. The recurrent neural network generally comprises an input layer, an output layer and a hidden layer, wherein the parameter U, V, W is shared in each input of the recurrent neural network, because the recurrent neural network is a recurrent system, parameters needing to be learned in the neural network are greatly reduced, and the hidden layer is most important for the recurrent neural network, is mainly used for capturing information of sequences and is used for realizing most of work. The main working principle is as follows:
Figure BDA0003165884770000071
Ot＝fo(Woutst)
in the formula:
Figure BDA0003165884770000072
Win、Woutall the matrixes are mainly used for connection and are respectively used for self-connection of nodes of a hidden layer, connection of an input layer to the hidden layer and weight connection of an output layer to the hidden layer; x is the number oftAnd OtDenoted as input and output of the t-th step, respectively, and the output of the hidden layer of the t-1 th step and the t-th step, respectively, is denoted as st-1And st，fsAnd foAre all activation functions and belong to the hidden layer and the output layer, respectively. The activation function is an important part of the structure for solving the nonlinear problem, and plays a great role in keeping the characteristics, removing redundant data and the like.
Taking a specific embodiment as an example, in order to reduce noise caused by a satellite instrument and facilitate input of a subsequent network model, denoising and slicing processing needs to be performed on a Black Body Temperature (TBB) and a clear air atmosphere rainfall (TPW) in a wind and cloud number 2 satellite.
In the wind and cloud No. 2 satellite, the equivalent blackbody brightness Temperature (TBB) is recorded once every 1 hour, the precipitation in clear air atmosphere (TPW) is recorded once every 3 hours, and information of the equivalent blackbody brightness temperature and the precipitation in clear air atmosphere can be memorized for a long time based on a recurrent neural network, so that time sequence characteristics of the equivalent blackbody brightness temperature and the precipitation in clear air atmosphere are extracted. The input of each sample in the recurrent neural network is a time sequence, and the time sequence characteristic of the T +1 moment is obtained according to the previous T moment.
S300, inputting the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution into a convolutional neural network, defining weight factors of loss functions in the convolutional neural network, and correcting the forecast data after model training to obtain a corrected precipitation result.
The Digital Elevation Model (DEM) is a solid ground Model which expresses the ground Elevation in the form of a group of ordered numerical value arrays, is a branch of the Digital Terrain Model (DTM), and can derive other various Terrain characteristic values.
The Convolutional Neural Networks (CNN) automatically extract important feature information of an image from the viewpoint of a Convolutional kernel and the combined features of neurons, and thus, the Convolutional Neural Networks can achieve a good effect in the field of image recognition. The convolutional neural network uses a group of elements distributed at different positions of the image but with the same weight vector to acquire the features of the image and form a feature map, and at each position, the units from different feature maps acquire different types of features, so that the spatial features of the two-dimensional data can be extracted. In the estimation of the rainfall field, the rainfall information of each point is related to the information around the point, and the spatial correlation between target grids of different perception domains and surrounding areas is automatically learned through a convolutional neural network and spatial feature information is obtained.
Specifically, different loss functions are designed for the problem of underestimating extreme precipitation, the data imbalance problem is caused due to the small proportion of the extreme precipitation, the network performance is reduced, and if the same weight is used, the network model underestimates the value of the extreme precipitation, so that the precipitation is corrected more accurately by adding a weight factor in the loss function for the extreme precipitation information.
In this embodiment, in the weight factor for defining the loss function in the convolutional neural network, the loss function L isestimationRelative error of plane rainfall LRE+ spatial correlation system LCCWherein, in the step (A),
Figure BDA0003165884770000081
Figure BDA0003165884770000082
wherein n is the total number of events, OiAnd SiI for the observed precipitation and the precipitation forecast to be tested,
Figure BDA0003165884770000083
and
Figure BDA0003165884770000084
the average values of the observed precipitation and the precipitation forecast are respectively.
Further, before the convolutional neural network corrects the forecast data, model training is performed by using the observed precipitation as a label, model training parameters are reserved, when the convolutional neural network corrects the forecast data, precipitation spatial distribution information is also obtained, a Vortex Pooling module (Vortex Pooling) is arranged in the convolutional neural network, and when the convolutional neural network obtains the precipitation spatial distribution information, context information is obtained through the Vortex Pooling module, specifically referring to fig. 4, where fig. 4 is a schematic diagram of the Vortex Pooling module in the embodiment of the present invention.
Context is a potential dependency relationship between an object and its surroundings, and is a key indicator for identifying an object. Because the targets present rich distribution relations under different scenes, the context characteristics are important information in the images, a network model is required to have an effective receptive field, and because the spatial information loss can be caused by multiple pooling operations in the convolutional neural network, certain small targets are easy to ignore in the characteristic extraction process, and the traditional pooling operations are downsampling in a certain pooling window with fixed size, the receptive field size is fixed, and the extracted context information is limited. And the vortex pooling module uses the expansion convolution, so that the parameters are unchanged and the characteristic diagram is reduced while the receptive field is increased, and in addition, the convolution with different expansion rates can extract the context information of different scales, thereby being beneficial to extracting the targets with different scales. And the spatial correlation is improved by describing local interaction of adjacent places to utilize the context information, so that the spatial distribution information of the precipitation can be better extracted.
S400, carrying out data inspection and evaluation on the correction precipitation result, wherein the data inspection and evaluation specifically comprises the following steps:
and S410, performing deterministic test and evaluation on the forecast capacity of correcting the precipitation result through a preset grading strategy.
Wherein, the predetermined scoring strategy comprises respectively calculating and evaluating a fair forecast rate (ETS), a Frequency deviation Frequency Bias (ETS), a hit rate (POD) and a False Alarm Rate (FAR), the fair forecast rate ETS refers to a rate at which events with random factors removed are evaluated to be correctly predicted, the Frequency deviation Frequency Bias refers to a rate at which events with occurrence of forecast and events with actual occurrence are evaluated (greater than 1 represents a high estimation of Frequency of occurrence of events and less than 1 represents a low estimation), the hit rate POD refers to a rate at which events with occurrence of evaluation are correctly predicted, the false alarm rate FAR refers to a rate at which events with no occurrence but with prediction are evaluated, and the optimal values thereof are 1, 1 and 1, 0, the variation ranges are-1/3-1, 0-1 and 0-infinity respectively, and the calculation formula is as follows:
Figure BDA0003165884770000091
Figure BDA0003165884770000092
Figure BDA0003165884770000093
Figure BDA0003165884770000101
wherein, H represents the number of correctly predicted lattice points, M represents the number of missed lattice points, F represents the number of empty lattice points, and C represents the number of correctly predicted lattice points without precipitation events, as shown in the following table.
Figure BDA0003165884770000102
S420, carrying out probabilistic test and evaluation on the forecasting capacity of correcting the precipitation result through probability operation, wherein the probabilistic test and evaluation specifically comprises the following steps:
s421, after the set dispersion of the set members in the precipitation result is compared with the mean root-mean-square error of the set, analyzing the error relation of the set dispersion to check the dispersion state of the set members,
will f isi(n) is reported as the predicted value of the nth set member of the ith sample, wherein i is 1,2,3, …, M; n-1, 2,3, …, N; m is the total number of samples, N is the set membership, andithe calculation formula is expressed as the observation of the ith sample, and is specifically as follows:
the ensemble average employed is that of,
Figure BDA0003165884770000103
the root mean square error is as follows,
Figure BDA0003165884770000104
the set of dispersion degrees is that,
Figure BDA0003165884770000105
for an ideal ensemble forecasting system, the magnitude of the ensemble dispersion and the magnitude of the root mean square error are the same, when the ensemble dispersion is smaller than the root mean square error, the system is in an under-dispersion state, and otherwise, the system is in an over-dispersion state.
And S422, drawing a working characteristic curve of the testee according to the calculation results of the hit rate and the false alarm rate to judge the forecasting skill.
A Receiver Operating Characteristic (ROC) curve, hereinafter referred to as ROC curve, is an image for describing sensitivity in a signal detection theory, and can be used for measuring the capability of a prediction system for distinguishing two classification events. Firstly, setting a probability threshold, considering the occurrence of the event according to the forecast probability which is more than or equal to the threshold, otherwise, not generating the event, converting the probability forecast into a common binary certainty forecast, calculating corresponding hit rate and false alarm rate, and specifically calculating by referring to the calculation formulas of the hit rate and the false alarm rate.
The false alarm rate and the hit rate in the binary classification certainty forecast are respectively used as the abscissa axis and the ordinate axis to establish a plane coordinate system, a series of points are drawn on the plane coordinate system according to the coordinate values of the false alarm rate and the hit rate, and a curve obtained by connecting the series of points is an ROC curve, wherein the closer the ROC curve is to the upper X axis and the left Y axis, the higher the forecast skill is. The area ROC area under the relative action characteristic curve is also an important index of mode test, the closer the ROC area is to 1, the higher the forecast skill is, and when the ROC area is less than or equal to 0.5, the no forecast skill is available.
S423, performing BSs score calculation based on the BS score result to compare the forecast probability and the actual occurrence probability, wherein,
will PiRecord as the forecast probability of sample i, let O beiAnd (3) recording as the actual occurrence probability of the sample i, wherein the calculation formula is as follows:
the BS score is such that,
Figure BDA0003165884770000111
Figure BDA0003165884770000112
the last three parts of the formula represent reliability, resolution, and uncertainties, respectively;
the BSS score is such that,
Figure BDA0003165884770000113
the Brier Skill Score (BSS) is based on the Brier Score (BS), taking into account the weather frequency of the sample, and comparing the predicted probability and the actual probability of occurrence of an event. For perfect forecast, BSS is 1, BSS greater than 0 indicates probabilistic forecast of skills, and BSS less than or equal to 0 indicates no skills.
As a specific embodiment, as shown in fig. 5, fig. 5 shows a time chart of the forecast rainfall and the observed rainfall before and after the deep learning correction, and as for the time chart of the rainfall data of 6-8 months from 2010 to 2012 in the Huaihe river basin, when the forecast is aged for one day, the accuracy of the integrated average rainfall result of the deep learning correction is higher than that of the original forecast rainfall, and the precipitation correction can be effectively performed. In the specific embodiment, the rainfall data of the Huaihe river basin from 2012 to 6-8 months is used for research, wherein the 2000-one 2009 data is used for training the deep learning model, and the 2010-one 2012 data is used as a test set to test the rainfall correction capability of the deep learning method.
The mean Root Mean Square Error (RMSE), Relative Error (RE) and spatial Correlation Coefficient (CC) of the ensemble averaged deep learning corrections are compared as shown in the following table:
Figure BDA0003165884770000121
in order to solve the above technical problem, an embodiment of the present invention further provides a rainfall forecast correcting system based on deep learning, as shown in fig. 6, including:
the system comprises a preprocessing unit, a data processing unit and a data processing unit, wherein the preprocessing unit is used for preprocessing forecast data of observed rainfall and mode output rainfall so as to enable the forecast data to have the same space-time resolution, and the forecast data comprise rainfall error distribution characteristics and influence characteristics on runoff simulation;
the circulating neural network is used for receiving satellite data which can respectively represent water vapor characteristics and vertical motion characteristics so as to extract time sequence characteristics of the satellite data;
the convolutional neural network is used for receiving the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution, and defining the weight factor of the loss function so as to correct the forecast data after model training to obtain a corrected precipitation result; and the number of the first and second groups,
and the inspection and evaluation unit is used for performing data inspection and evaluation on the corrected precipitation result.
In particular, in a convolutional neural network, the loss function LestimationRelative error of plane rainfall LRE+ spatial correlation system LCCWherein, in the step (A),
Figure BDA0003165884770000122
Figure BDA0003165884770000123
wherein n is the total number of events, OiAnd SiI for the observed precipitation and the precipitation forecast to be tested,
Figure BDA0003165884770000131
and
Figure BDA0003165884770000132
the average values of the observed precipitation and the precipitation forecast are respectively.
Furthermore, before the convolutional neural network corrects the forecast data, model training is carried out by taking the observed rainfall as a label, model training parameters are reserved, when the convolutional neural network corrects the forecast data, rainfall spatial distribution information is also obtained, a Vortex Pooling module (Vortex Pooling) is arranged in the convolutional neural network, and when the convolutional neural network obtains the rainfall spatial distribution information, context information is obtained through the Vortex Pooling module.
The embodiment of the invention combines a physical mechanism, utilizes a cyclic neural network model to extract the time sequence characteristics of water vapor and vertical movement in the precipitation forming condition, then inputs elevation data into a convolution neural network model together based on the influence of terrain on precipitation, and the depth of the model combines the weather principle and an artificial intelligence model, firstly utilizes the cyclic neural network to extract the physical characteristics, and then inputs the physical characteristics into the model structure of the convolution neural network, thereby pertinently improving the precision of rainstorm magnitude precipitation and the spatial distribution of precipitation and effectively improving the precipitation performance; meanwhile, based on prior information such as influence of rainfall error distribution on runoff simulation, an appropriate deep learning model is selected for improving the runoff simulation, the problem of insufficient extraction of extreme rainfall forecast information is solved by emphatically improving rainfall spatial distribution through increasing context information, improving weight and the like, a large amount of levels of rainfall is emphatically corrected by adjusting a loss function and a weight factor, and the problem of general underestimation of large-value rainfall is solved.
In order to solve the above technical problem, an embodiment of the present invention further provides a computer device. Referring to fig. 7, fig. 7 is a block diagram of a basic structure of a computer device according to the present embodiment.
Fig. 7 is a schematic diagram of the internal structure of the computer device. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The non-volatile storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store control information sequences, and the computer readable instructions can enable the processor to realize a rainfall forecast correction method when being executed by the processor. The processor of the computer device is used for providing calculation and control capability and supporting the operation of the whole computer device. The memory of the computer device may have computer readable instructions stored therein which, when executed by the processor, may cause the processor to perform a method of correcting precipitation forecasts. The network interface of the computer device is used for connecting and communicating with the terminal. Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In this embodiment, the processor is configured to execute specific functions of the preprocessing unit, the recurrent neural network, the convolutional neural network, and the inspection and evaluation unit in fig. 6, and the memory stores program codes and various types of data required for executing the above modules. The network interface is used for data transmission to and from a user terminal or a server. The memory in this embodiment stores program codes and data necessary for executing all the submodules, and the server can call the program codes and data of the server to execute the functions of all the submodules.
The present invention also provides a storage medium storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the precipitation forecast correction method according to any of the embodiments described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

Claims (10)

1. A rainfall forecast correction method based on deep learning is characterized by comprising the following steps:
preprocessing forecast data of observed rainfall and mode output rainfall by using a space-time interpolation method so as to enable the forecast data to have the same space-time resolution;
inputting satellite data which can respectively represent water vapor characteristics and vertical motion characteristics into a circulating neural network to extract time sequence characteristics of the satellite data;
inputting the forecast data, the time sequence characteristics and DEM data with the same space-time resolution into a convolutional neural network, defining weight factors of loss functions in the convolutional neural network, and correcting the forecast data after model training to obtain a corrected precipitation result;
and carrying out data inspection and evaluation on the correction precipitation result.
2. The deep learning-based rainfall forecast correction method of claim 1, wherein said forecast data comprises rainfall error distribution characteristics and impact characteristics on runoff simulation.
3. The deep learning-based rainfall forecast correction method of claim 2, wherein said weight factors defining loss functions in convolutional neural network, said loss functions LestimationRelative error of plane rainfall LRE+ spatial correlation system LCCWherein, in the step (A),
Figure FDA0003165884760000011
Figure FDA0003165884760000012
wherein n is the total number of events, OiAnd SiI for the observed precipitation and the precipitation forecast to be tested,
Figure FDA0003165884760000013
and
Figure FDA0003165884760000014
the average values of the observed precipitation and the precipitation forecast are respectively.
4. The deep learning-based rainfall forecast correction method of claim 3, wherein when said convolutional neural network corrects said forecast data, rainfall spatial distribution information is also obtained.
5. The deep learning-based rainfall forecast correcting method according to claim 4, wherein a vortex pooling module is arranged in the convolutional neural network, and when the convolutional neural network obtains the rainfall spatial distribution information, context information is obtained through the vortex pooling module.
6. The deep learning-based rainfall forecast correction method of claim 5, wherein before the convolutional neural network corrects the forecast data, model training is performed with the observed rainfall as a label and model training parameters are retained.
7. The deep learning-based precipitation forecast correcting method according to claim 1, wherein the data verification and evaluation of the corrected precipitation result specifically comprises:
carrying out deterministic test and evaluation on the forecasting capacity of correcting the precipitation result through a preset scoring strategy;
and carrying out probabilistic test and evaluation on the forecasting capacity for correcting the precipitation result through probability operation.
8. The deep learning-based rainfall forecast correcting method according to claim 7, wherein the predetermined scoring strategy comprises respectively performing calculation and evaluation on a fair forecast rate ETS, a Frequency deviation Frequency Bias, a hit rate POD and a false alarm rate FAR, and the calculation formula is as follows:
Figure FDA0003165884760000021
Figure FDA0003165884760000022
Figure FDA0003165884760000023
Figure FDA0003165884760000024
h represents the number of grid points correctly forecasted, M represents the number of missed report grid points, F represents the number of empty report grid points, and C represents the number of grid points correctly forecasted without precipitation events.
9. The deep learning-based precipitation forecast correcting method according to claim 8, wherein the probabilistic verification and evaluation of the forecast capability of correcting precipitation results through probability calculation specifically comprises:
after comparing the set dispersion of the set members in the corrected precipitation result with the mean root-mean-square error of the set, analyzing the error relation of the set dispersion to check the dispersion state of the set members,
will f isi(n) is reported as the predicted value of the nth set member of the ith sample, wherein i is 1,2,3, …, M; n-1, 2,3, …, N; m is the total number of samples and N is the set membership, williThe calculation formula is expressed as the observation of the ith sample, and is specifically as follows:
the ensemble average employed is that of,
Figure FDA0003165884760000025
the root mean square error is as follows,
Figure FDA0003165884760000026
the set of dispersion degrees is that,
Figure FDA0003165884760000031
drawing a working characteristic curve of the testee according to the calculation results of the hit rate and the false alarm rate to judge the forecasting skill;
performing a BSs score calculation based on the BS score result to compare the forecast probability and the actual occurrence probability, wherein,
will PiRecord as the forecast probability of sample i, let O beiAnd (3) recording as the actual occurrence probability of the sample i, wherein the calculation formula is as follows:
the BS score is such that,
Figure FDA0003165884760000032
Figure FDA0003165884760000033
the BSS score is such that,
Figure FDA0003165884760000034
10. a rainfall forecast correction system based on deep learning, comprising:
the preprocessing unit is used for preprocessing forecast data of observed rainfall and mode output rainfall so as to enable the forecast data to have the same space-time resolution;
the circulating neural network is used for receiving satellite data which can respectively represent water vapor characteristics and vertical motion characteristics so as to extract time sequence characteristics of the satellite data;
the convolutional neural network is used for receiving the forecast data, the time sequence characteristics and the DEM data with the same space-time resolution, and defining the weight factor of the loss function so as to correct the forecast data after model training to obtain a corrected precipitation result; and the number of the first and second groups,
and the inspection and evaluation unit is used for performing data inspection and evaluation on the corrected precipitation result.
CN202110804467.1A 2021-07-16 2021-07-16 Precipitation prediction correction method and system based on deep learning Active CN113496104B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110804467.1A CN113496104B (en) 2021-07-16 2021-07-16 Precipitation prediction correction method and system based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110804467.1A CN113496104B (en) 2021-07-16 2021-07-16 Precipitation prediction correction method and system based on deep learning

Publications (2)

Publication Number Publication Date
CN113496104A true CN113496104A (en) 2021-10-12
CN113496104B CN113496104B (en) 2024-03-22

Family

ID=77996175

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110804467.1A Active CN113496104B (en) 2021-07-16 2021-07-16 Precipitation prediction correction method and system based on deep learning

Country Status (1)

Country Link
CN (1) CN113496104B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962465A (en) * 2021-10-26 2022-01-21 国家气象中心(中央气象台) Precipitation forecasting method, equipment, device and storage medium
CN114020725A (en) * 2021-11-11 2022-02-08 中国水利水电科学研究院 Window sliding GPM data correction method considering spatial distribution
CN114611608A (en) * 2022-03-14 2022-06-10 中国人民解放军国防科技大学 Sea surface height numerical value prediction deviation correction method based on deep learning model
CN114624791A (en) * 2022-03-18 2022-06-14 杭州鲁尔物联科技有限公司 Rainfall measurement method and device, computer equipment and storage medium
CN114676641A (en) * 2022-04-16 2022-06-28 江苏省气象台 Rainfall probability forecasting algorithm based on heterogeneous regression variable sliding training period
CN115047543A (en) * 2022-08-16 2022-09-13 成都信息工程大学 Power transmission corridor rainfall early warning method and system
CN115690607A (en) * 2023-01-04 2023-02-03 西湖大学 Rainfall inversion method and device based on infrared remote sensing and deep learning
CN116467946A (en) * 2023-04-21 2023-07-21 南京信息工程大学 Deep learning-based mode prediction product downscaling method
WO2023240509A1 (en) * 2022-06-15 2023-12-21 中山大学 Spatial probability analysis method and system based on correspondence between precipitation forecast and teleconnection
CN117388953A (en) * 2023-12-13 2024-01-12 南京信息工程大学 Weather forecast method for improving MIM-rwkv by SADBO based on big data frame

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7542852B1 (en) * 2005-01-25 2009-06-02 Weather Channel Inc Derivation and production of high-resolution, very short-term weather forecasts
CN110472762A (en) * 2018-05-11 2019-11-19 中国电力科学研究院有限公司 A kind of correction method and system of gridding precipitation
CN111105068A (en) * 2019-11-01 2020-05-05 复旦大学 Numerical value mode correction method based on sequence regression learning
CN111257970A (en) * 2018-11-30 2020-06-09 中国电力科学研究院有限公司 Rainfall forecast correction method and system based on ensemble forecast
CN111260111A (en) * 2020-01-08 2020-06-09 中国科学院上海技术物理研究所苏州研究院 Runoff forecast improvement method based on meteorological big data
CN112215393A (en) * 2020-08-29 2021-01-12 复旦大学 Rainfall numerical prediction post-processing correction method based on adaptive space-time scale selection
CN112213727A (en) * 2020-10-15 2021-01-12 国家卫星气象中心(国家空间天气监测预警中心) Precipitation correction method of satellite-borne radar based on active and passive microwave combined detection

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7542852B1 (en) * 2005-01-25 2009-06-02 Weather Channel Inc Derivation and production of high-resolution, very short-term weather forecasts
CN110472762A (en) * 2018-05-11 2019-11-19 中国电力科学研究院有限公司 A kind of correction method and system of gridding precipitation
CN111257970A (en) * 2018-11-30 2020-06-09 中国电力科学研究院有限公司 Rainfall forecast correction method and system based on ensemble forecast
CN111105068A (en) * 2019-11-01 2020-05-05 复旦大学 Numerical value mode correction method based on sequence regression learning
CN111260111A (en) * 2020-01-08 2020-06-09 中国科学院上海技术物理研究所苏州研究院 Runoff forecast improvement method based on meteorological big data
CN112215393A (en) * 2020-08-29 2021-01-12 复旦大学 Rainfall numerical prediction post-processing correction method based on adaptive space-time scale selection
CN112213727A (en) * 2020-10-15 2021-01-12 国家卫星气象中心(国家空间天气监测预警中心) Precipitation correction method of satellite-borne radar based on active and passive microwave combined detection

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HOANG-MINH NGUYEN ET AL.: "An approach for improving the capability of a coupled meteorological and hydrological model for rainfall and flood forecasts", 《JOURNAL OF HYDROLOGY》, vol. 577 *
苏翔 等: "四种定量降水预报客观订正方法对比研究", 《气象学报》, vol. 79, no. 01 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113962465A (en) * 2021-10-26 2022-01-21 国家气象中心(中央气象台) Precipitation forecasting method, equipment, device and storage medium
CN114020725A (en) * 2021-11-11 2022-02-08 中国水利水电科学研究院 Window sliding GPM data correction method considering spatial distribution
CN114611608A (en) * 2022-03-14 2022-06-10 中国人民解放军国防科技大学 Sea surface height numerical value prediction deviation correction method based on deep learning model
CN114611608B (en) * 2022-03-14 2024-04-09 中国人民解放军国防科技大学 Sea surface height numerical value forecast deviation correction method based on deep learning model
CN114624791A (en) * 2022-03-18 2022-06-14 杭州鲁尔物联科技有限公司 Rainfall measurement method and device, computer equipment and storage medium
CN114676641A (en) * 2022-04-16 2022-06-28 江苏省气象台 Rainfall probability forecasting algorithm based on heterogeneous regression variable sliding training period
CN114676641B (en) * 2022-04-16 2024-05-14 江苏省气象台 Precipitation probability prediction algorithm based on non-homogeneous regression variable sliding training period
WO2023240509A1 (en) * 2022-06-15 2023-12-21 中山大学 Spatial probability analysis method and system based on correspondence between precipitation forecast and teleconnection
CN115047543A (en) * 2022-08-16 2022-09-13 成都信息工程大学 Power transmission corridor rainfall early warning method and system
CN115047543B (en) * 2022-08-16 2022-11-11 成都信息工程大学 Power transmission corridor rainfall early warning method and system
CN115690607B (en) * 2023-01-04 2023-04-11 西湖大学 Rainfall inversion method and device based on infrared remote sensing and deep learning
CN115690607A (en) * 2023-01-04 2023-02-03 西湖大学 Rainfall inversion method and device based on infrared remote sensing and deep learning
CN116467946B (en) * 2023-04-21 2023-10-27 南京信息工程大学 Deep learning-based mode prediction product downscaling method
CN116467946A (en) * 2023-04-21 2023-07-21 南京信息工程大学 Deep learning-based mode prediction product downscaling method
CN117388953A (en) * 2023-12-13 2024-01-12 南京信息工程大学 Weather forecast method for improving MIM-rwkv by SADBO based on big data frame
CN117388953B (en) * 2023-12-13 2024-03-08 南京信息工程大学 Weather forecast method for improving MIM-rwkv by SADBO based on big data frame

Also Published As

Publication number Publication date
CN113496104B (en) 2024-03-22

Similar Documents

Publication Publication Date Title
CN113496104B (en) Precipitation prediction correction method and system based on deep learning
Zhou et al. Forecasting different types of convective weather: A deep learning approach
Kumar et al. Convcast: An embedded convolutional LSTM based architecture for precipitation nowcasting using satellite data
US11488069B2 (en) Method for predicting air quality with aid of machine learning models
Veldkamp et al. Statistical postprocessing of wind speed forecasts using convolutional neural networks
CN112598248B (en) Load prediction method, load prediction device, computer equipment and storage medium
CN111665575B (en) Medium-and-long-term rainfall grading coupling forecasting method and system based on statistical power
Olsson et al. Statistical atmospheric downscaling of short-term extreme rainfall by neural networks
CN117556197B (en) Typhoon vortex initialization method based on artificial intelligence
CN114611608A (en) Sea surface height numerical value prediction deviation correction method based on deep learning model
CN115267945A (en) Thunder and lightning early warning method and system based on graph neural network
Wang et al. Nowcasting multicell short-term intense precipitation using graph models and random forests
CN114882373A (en) Multi-feature fusion sandstorm prediction method based on deep neural network
Wang et al. Relative importance of radar variables for nowcasting heavy rainfall: A machine learning approach
CN113269805A (en) Remote sensing rainfall inversion training sample self-adaptive selection method guided by rainfall event
CN113033910A (en) Photovoltaic power generation power prediction method, storage medium and terminal equipment
CN117494871A (en) Ship track prediction method considering ship interaction influence
Luo et al. Robust precipitation bias correction through an ordinal distribution autoencoder
Rossi et al. Analysis of a statistically initialized fuzzy logic scheme for classifying the severity of convective storms in F inland
CN116679355A (en) Precipitation prediction correction method based on cascade Attention-U-Net
CN115980885A (en) Rainfall forecast deviation correction method based on ensemble forecast
Danpoonkij et al. Incorporating prior scientific knowledge into deep learning for precipitation nowcasting on radar images
Miralles et al. Bayesian modeling of insurance claims for hail damage
Tao et al. Error variance estimation for individual geophysical parameter retrievals
Socaci et al. XNow: A deep learning technique for nowcasting based on radar products’ values prediction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant