CN114137541A - Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB - Google Patents
Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB Download PDFInfo
- Publication number
- CN114137541A CN114137541A CN202111374147.3A CN202111374147A CN114137541A CN 114137541 A CN114137541 A CN 114137541A CN 202111374147 A CN202111374147 A CN 202111374147A CN 114137541 A CN114137541 A CN 114137541A
- Authority
- CN
- China
- Prior art keywords
- layer
- radar echo
- point
- module
- sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003860 storage Methods 0.000 title claims abstract description 10
- 238000000605 extraction Methods 0.000 claims abstract description 21
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims abstract description 12
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 239000000284 extract Substances 0.000 claims abstract description 5
- 238000010606 normalization Methods 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 16
- 238000013507 mapping Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 230000015654 memory Effects 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 description 7
- 125000004122 cyclic group Chemical group 0.000 description 5
- 238000013213 extrapolation Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000000306 recurrent effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 241001061260 Emmelichthys struhsakeri Species 0.000 description 1
- 241000276495 Melanogrammus aeglefinus Species 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 230000003042 antagnostic effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013277 forecasting method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 239000011541 reaction mixture Substances 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/95—Radar or analogous systems specially adapted for specific applications for meteorological use
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S7/411—Identification of targets based on measurements of radar reflectivity
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S7/417—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section involving the use of neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Electromagnetism (AREA)
- Radar Systems Or Details Thereof (AREA)
Abstract
The invention discloses a method, a device and a storage medium for forecasting short rainfall on the basis of Transformer-IRB, wherein the method comprises the following steps: preprocessing the radar echo image, sending the preprocessed continuous M frames of radar echo images into a pre-constructed and trained short rainfall prediction model based on a Transformer-IRB to obtain N frames of radar echo images, wherein the short rainfall prediction model comprises a block embedding module and a feature extraction module, the device comprises a decoding module, a block embedding module, a feature extraction module and a decoding module, wherein the block embedding module extracts embedded features of continuous multi-frame radar echo graphs and sends the embedded features into the feature extraction module after adding position codes, the feature extraction module adopts a plurality of transducer-inverted residual block modules with multi-head attention layers to form an encoder to extract the features of the input embedded features and then send the extracted features into the decoding module, and the decoding module decodes the input to obtain the continuous multi-frame radar echo graphs.
Description
Technical Field
The invention relates to the field of weather forecast, in particular to a method, a device and a storage medium for forecasting short rainfall based on Transformer-IRB.
Background
The short rainfall forecast refers to rainfall forecast of 0-6 hours in the future, is an important problem in the field of weather forecast, can provide weather guidance for daily life and professional production activities of the public, and plays an important role in promoting economic development, protecting life and property safety of the public and the like. The existing short rainfall forecasting methods mainly comprise two types, one type is a radar echo map extrapolation method based on an optical flow method, and the other type is a radar echo map extrapolation method based on machine learning.
The Radar echo graph extrapolation method based on the Optical flow method is to estimate the motion mode of convection cloud from continuous Radar echo graphs and then predict a future Radar echo graph by using a semi-lagrange advection model, such as a Real-time Optical flow by variable methods for echo of Radar used by hong kong astronomical instruments of china.
Because the radar echo diagram extrapolation method based on the optical flow method is established on the assumption that smooth motion and total rainfall intensity are not changed, high-precision rainfall prediction is difficult to perform when dynamic and nonlinear motion modes and rainfall intensity scenes with rapid changes are encountered. Therefore, a machine learning method for modeling a radar echo map sequence space-time process by using a deep neural network model with strong fitting nonlinear transformation capability and driving a large amount of historical data is receiving more and more attention, and the work can be divided into two types, namely a cyclic neural network architecture and a non-cyclic neural network architecture. In the study based on the recurrent neural network, the representative work is as the Convolitional LSTM (Long Short-Term Memory) model proposed by Shi et al in 2015, the model uses the recurrent neural network to estimate the hidden state corresponding to the radar echo diagram sequence on the basis of the LSTM, and the recurrent neural network introduces the learning of local spatial correlation, so that the model achieves higher prediction accuracy compared with the models such as ROVER and FC-LSTM. However, it is difficult to capture the variable morphology of the convective cloud by performing a convolution operation on each point by using a fixed sampling template in the conventional convolutional neural network, so that Shi et al in 2017 proposes a trajgru (projective Gated secure unit) model, which infers the sampling point position of each point by using the convolutional neural network to perform resampling, thereby obtaining an unfixed sampling point from global input information to capture the variable spatial morphology features. In the work of the non-cyclic neural network structure, for example, a CNN-based method proposed by Ayzel et al in 2019 is represented, and a U-Net architecture is adopted to construct a prediction model. An antagonistic network generation method is also concerned, such as an artificial intelligence forecasting model constructed based on a GAN method and proposed by Shenzhen gas bureau in 2019 in combination with the Harmony project, and such as a short rainfall forecasting model proposed by Tianan in 2019 and based on ConvGRU and GAN methods. In the work, the model based on the recurrent neural network architecture has great limitation in capturing the space-time dependence in a larger range; the model space-time feature extraction capability based on the non-cyclic neural network architecture still needs to be improved, the attention to the global space-time feature relation is lacked, and the accuracy of the short-term rainfall forecasting model is restricted.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a method, an apparatus and a storage medium for predicting short rainfall based on Transformer-IRB, aiming at the above-mentioned defects in the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows:
in one aspect, a method for predicting short rainfall on the basis of Transformer-IRB is constructed, and comprises the following steps:
preprocessing a radar echo image;
sending the preprocessed continuous M frames of radar echo images into a pre-constructed and trained short rainfall prediction model based on a transform-IRB to obtain N frames of radar echo images as a short rainfall prediction result, wherein N is larger than M, the short rainfall prediction model comprises a block embedding module, a feature extraction module and a decoding module, the block embedding module extracts the embedding features of continuous multi-frame radar echo images and sends the embedding features into the feature extraction module after adding position codes, the feature extraction module adopts a plurality of transform-inversion residual block modules with multi-head attention layers to form an encoder to carry out feature extraction on the input embedding features and then sends the encoder into the decoding module, and the decoding module decodes the input to obtain the continuous multi-frame radar echo images as the prediction result.
Preferably, the preprocessing the radar echo image includes: mapping pixel values of the radar echo image to a range of [0,255 ];
the method further comprises the following steps: and acquiring a de-noising mask in advance according to historical data of the radar echo image, and performing point multiplication on the radar echo image output by the short rainfall prediction model and the de-noising mask to obtain a de-noised radar echo image.
Preferably, the acquiring the denoising mask includes:
defining a denoising mask with the same resolution as the radar echo image, and initializing pixel values of all pixels of the denoising mask to be 1;
searching all pixels of the radar echo map, and recording the pixels of which the Mahalanobis distance of the pixel values is more than 3 times of the standard deviation as noise points;
marking pixels with pixel values larger than 0 and smaller than 70 in the radar echo chart as noise points;
and updating the pixel value of a pixel in the denoising mask, which is the same as the position of the noise point, to be 0.
Preferably, the block embedding module includes two layers of point-by-point convolutional layers and a graph-to-sequence layer, the output of the first convolutional layer is sent to the second convolutional layer after being nonlinearly mapped by using an activation function, the output of the second convolutional layer is sent to the graph-to-sequence layer after being linearly mapped, and the graph-to-sequence layer converts the feature graph processed by the second convolutional layer into a feature sequence according to a spatial position.
Preferably, the transform-inverse residual block module comprises a multi-head attention layer, two layer normalization layers, a sequence-to-layer, two point-by-point convolution layers, a depth separable convolution layer, and a graph-to-sequence layer;
the transform-inverse residual block module receives the characteristic embedding output by the last module and sends the characteristic embedding into a first layer normalization layer, the output of the first layer normalization layer is sequentially processed by the multi-head attention layer, processed by a second layer normalization layer and sequenced to the layer, the first point-by-point convolution layer is sent into the depth separable convolution layer, the output of the first point-by-point convolution layer and the output of the depth separable convolution layer are added and then sent into the second point-by-point convolution layer, the output of the second point-by-point convolution layer is sent into the sequence layer, and the output of the sequence layer from the graph and the input of the first layer normalization layer are added and then sent into the next module;
the first point-by-point convolution layer and the depth separable convolution layer adopt a GeLU activation function to carry out nonlinear mapping, the second point-by-point convolution layer carries out linear mapping, and the embedded characteristic sequence is converted into a characteristic diagram from the sequence to the layer according to the spatial position.
Preferably, the point-by-point convolution layer is a convolution layer or a deconvolution layer with the grouping number of 1; the depth separable convolutional layer is a convolutional layer or a deconvolution layer with the number of packets as the output dimension.
Preferably, the decoding module is sequentially composed of a sequence to a layer and two deconvolution neural network layers, and embeds and maps the features into a multi-frame radar echo map.
Preferably, the method further comprises:
before the short rainfall prediction model is put into use, randomly extracting continuous M + N frames of denoised radar echo images from historical data to randomly train the short rainfall prediction model;
after the short rainfall prediction model is trained randomly and put into use, continuous M + N frames of radar echo maps are extracted from the latest historical data in a sliding mode through a window with the size of M frames for sequential training.
In a second aspect, a transform-IRB based short-rainfall prediction device is constructed, comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, carries out the steps of the method according to any of the preceding claims.
In three aspects, a storage medium is constructed storing a computer program which, when executed by a processor, implements the steps of the method as claimed in any one of the preceding claims.
The method, the device and the storage medium for predicting the short rainfall on the basis of the Transformer-IRB have the following beneficial effects: the method combines a self-attention mechanism and an inverted residual error network structure to capture space-time information in a wider range, adopts the inverted residual error network structure with stronger feature learning capability to extract complex space-time features in a radar echo diagram sequence, and adopts a self-attention mechanism to capture long-range global space-time dependence, so that the rainfall evolution process of a long-range space-time range in the radar echo diagram sequence can be better fitted, and the prediction precision of the short-term rainfall is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts:
FIG. 1 is a flow chart of a method for forecasting short rainfall based on Transformer-IRB according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a short rainfall prediction model;
FIG. 3 is a schematic diagram of a block embedding module;
FIG. 4 is a schematic structural diagram of a transform-inverse residual block module.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Exemplary embodiments of the invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
The general idea of the invention is as follows: preprocessing the radar echo images, and sending the preprocessed continuous M frames of radar echo images into a pre-constructed and trained short rainfall prediction model based on a transform-IRB to obtain N frames of radar echo images as a short rainfall prediction result; the short rainfall prediction model comprises a block embedding module, a feature extraction module and a decoding module, wherein the block embedding module extracts embedded features of continuous multi-frame radar echo graphs and sends the embedded features into the feature extraction module after position coding is added, the feature extraction module adopts a plurality of transducer-inverted residual block modules with multi-head attention layers to form an encoder to extract features of input embedded features and then sends the features into the decoding module, and the decoding module decodes the input to obtain continuous multi-frame radar echo graphs as prediction results.
In order to better understand the technical solutions, the technical solutions will be described in detail below with reference to the drawings and the specific embodiments of the specification, and it should be understood that the embodiments and specific features of the embodiments of the present invention are detailed descriptions of the technical solutions of the present application, and are not limited to the technical solutions of the present application, and the technical features of the embodiments and examples of the present invention may be combined with each other without conflict.
Example one
Referring to fig. 1, the method for predicting short rainfall based on Transformer-IRB of the present embodiment includes:
s100: constructing and training a short rainfall prediction model, and acquiring a denoising mask;
the input of the short rainfall prediction model is M frames of radar echo images, the output is N frames of radar echo images, M, N are positive integers, and N is larger than M. Specifically, referring to fig. 2, the short rainfall prediction model includes a block Embedding (Patch Embedding) module, a Feature Extraction (Feature Extraction) module, and a Decoding (Decoding) module. The Block embedding module extracts embedding characteristics of continuous multi-frame radar echo diagrams, adds position codes to the embedding characteristics and sends the embedding characteristics to the characteristic extraction module, the characteristic extraction module adopts a plurality of (for example, k Inverted Residual modules in fig. 2, k is a positive integer) transducer-Inverted Residual blocks (transducer-IRB) with multi-head attention layers to form an encoder, performs characteristic extraction on input embedding characteristics and sends the extracted embedding characteristics to the decoding module, and the decoding module decodes the input embedding characteristics to obtain the continuous multi-frame radar echo diagrams as prediction results.
Referring to fig. 3, the block embedding module includes two layers of Point-wise Convolution (Point-wise Convolution) and Image to Sequence (Image to Sequence), where an output of a first convolutional layer is sent to a second convolutional layer after being nonlinearly mapped by using an activation function (GeLU), an output of the second convolutional layer is sent to the Image to Sequence layer after being linearly mapped, and no nonlinear mapping is performed to retain more information, and the Image to Sequence layer converts a feature map processed by the second convolutional layer into a feature Sequence (i.e., converts the feature map into a feature Sequence according to a spatial position) (i.e., the feature Sequence is generated by the Image to Sequence layer)
). The two layers of point-by-point convolution can reduce the resolution and increase the effective receptive field of the space convolution operation of the subsequent module. In the present invention, the character F is the feature map, C is the number of feature map channels, H is the width of the feature map, W is the height of the feature map, and L is the length of the sequence.
Referring to fig. 4, each of the fransformer-inverse residual block modules includes a Multi-head Attention (Multi-head association) Layer, two Layer Normalization layers (Layer Normalization), a Sequence to Image (Sequence to Image) Layer, two Point-wise Convolution layers (Point-wise Convolution) layers, a Depth-wise Convolution Layer (Depth-wise Convolution) Layer, and a Sequence to Image (Sequence to Image) Layer.
The embedded characteristic of the output of the last module received by the transform-inverse residual block module is sent to a first layer normalization layer for processing, the output of the first layer normalization layer is sent to a depth separable convolution layer sequentially through the multi-head attention layer, a second layer normalization layer, a sequence-to-layer and a first point-to-point convolution layer, the output of the first point-to-point convolution layer and the output of the depth separable convolution layer are added and then sent to the second point-to-point convolution layer, the output of the second point-to-point convolution layer is sent to a sequence layer, and the output of the graph-to-sequence layer and the input of the first layer normalization layer are added and then sent to the next module as the final output of the transform-inverse residual block module.
The first point-by-point convolution layer and the depth separable convolution layer adopt an activation function (GeLU) to carry out nonlinear mapping, the second point-by-point convolution layer carries out linear mapping without adopting the activation function to carry out nonlinear mapping so as to reserve more information, and the sequence is converted into a characteristic diagram (namely the sequence is converted into the layer according to the spatial position) from the sequence to the layer
Different from a standard inverted residual error module, the Transformer-inverted residual error block module disclosed by the invention is combined with the Transformer module and an inverted residual error block network structure, and firstly, a layer normalization layer, a multi-head attention layer and a layer normalization layer are embedded into input features and sequentially processed; in addition, jump connections are added at the depth-separable convolutional layer, i.e.
The decoding module is sequentially composed of a sequence to a layer and two deconvolution neural network layers, and embeds and maps the features into a multi-frame radar echo map.
In the above model design, the convolutional layer (or deconvolution layer) design parameters of the model are as shown in table 1, and the point-by-point convolutional layer is a convolutional layer (or deconvolution layer) having the number of groups (groups) of 1, and the depth separable convolutional layer is a convolutional layer (or deconvolution layer) having the number of groups (groups) of output dimensions.
TABLE 1 model convolution layer (or deconvolution layer) design parameters
The multi-head attention layer comprises two fully-connected network layers and one Dropout layer, wherein the fully-connected network layer is calculated according to the formula (1), dheadHead for attentioniAnd the two fully-connected layers do not use the activation function, the output of the second fully-connected layer is processed by the Dropout layer.
MultiHead(y)＝Concat(head1，...，headh)WO (1)；
In the formula (1), the reaction mixture is,
embedded features (embedding); concat (·) denotes the stitching of the outputs of the h attention heads in the feature dimension;
is a learnable parameter matrix;
wherein, Wi Q，Wi K，
Is a learnable parameter matrix (i is more than or equal to 1 and less than or equal to h); dhead＝dmodelH is the number of attention heads; qi＝yWi Q，Ki＝yWi K，Vi＝yWi V(i is more than or equal to 1 and less than or equal to h); softmax (·) denotes an activation function
softmax (·) j is computed and then processed through a Dropout layer.
With regard to the training of the model, it is carried out in two phases: in the first stage, before the short rainfall prediction model is used for the first time, continuous M + N frames of denoised radar echo images are randomly extracted from historical data to train the short rainfall prediction model. After the model is put into use, the model may be trained on line in real time by using the latest generated historical data to ensure the accuracy of the model, and therefore, the method of this embodiment further includes: after the short rainfall prediction model is put into use, continuous M + N frames of radar echo maps are extracted from historical data in a window with the size of M frames in a sliding mode and are trained.
The loss function of model training adopts the sum of a balance mean square error formula (2) and a balance absolute error formula (3):
wherein N is the sequence frame number of the radar echo map, H, W is the height and width of the radar echo map, xn,i,jThe position of the radar echo diagram of the nth frame is the rainfall observation value of the pixel of (i, j),
the nth frame radar echo position is the rainfall prediction value of the pixel of (i, j), wn,i,jThe position of the nth radar echo map is the error weight of the pixel of (i, j), the weight calculation corresponding to each radar echo map is shown in formula (4) (the noise point position weight is 0), wherein x is rainfall intensity. Since the pixel value (pixel value) of the original radar echo map is the echo reflectivity value Z (db), it needs to be converted into the rainfall intensity R (mm/h), and the Z-R relationship is described in formula (5), where a is 58.53, b is 1.56:
Z＝10log a+10b log R (5)；
regarding the denoising mask, which is used for denoising an image output by a model, the acquisition of the denoising mask mainly includes:
1) acquiring historical data of a radar echo image, defining a denoising mask with the same resolution as the radar echo image, and initializing pixel values of all pixels of the denoising mask to be 1;
2) searching all pixels of the radar echo chart, and recording the pixels of which the Mahalanobis distance of the pixel values is more than 3 times of the standard deviation as noise points, wherein the related calculation formula is as follows:
wherein N represents the number of pixels of the radar echo image, xiPixel value, D, representing a pixel of a radar echo mapMahalanobis(x) Which represents the mahalanobis distance,
represents the mean value of the pixel values of the radar echo diagram, sigma represents the standard deviation,
representing a covariance matrix.
3) Marking pixels with pixel values larger than 0 and smaller than 70 in the radar echo chart as noise points;
4) and updating the pixel value of a pixel in the denoising mask, which is the same as the position of the noise point, to be 0.
Similarly, the denoising mask is obtained according to the historical data of the radar echo image, and can be stored after the denoising mask is obtained, and the image output by the model is denoised after the model is put into use, of course, considering that the historical data according to the radar echo image can be gradually accumulated and updated, the denoising mask can be updated on line by using the latest historical data as the model, and the updating process is to perform the steps 1) to 4) again by using the latest historical data.
Step S100 is a preparation work for obtaining a model, and thereafter, the following steps S101 to S104 may be performed to predict the short rainfall event using the newly acquired radar echo image and the model obtained in step S100.
S101: preprocessing a radar echo image;
before entering the image into the model, we need to pre-process the image, including: mapping pixel values of the radar echo image to a [0,255] range based on a calculation formula (10), and then carrying out denoising processing on the radar echo image:
wherein, dBZ represents the pixel value of the original radar echo image, and pixel represents the pixel value of the radar echo image after preprocessing.
S102: sending the preprocessed continuous M frames of radar echo images into a pre-constructed and trained short rainfall prediction model based on a Transformer-IRB to obtain N frames of radar echo images as a short rainfall prediction result;
s103: and performing point multiplication on the radar echo image output by the short rainfall prediction model and a denoising mask to obtain a denoised radar echo image.
Next, we performed experiments with 20 predicted frames of input 5 frames, and the comparison results are shown in tables 2, 3, and 4, and the model experiments of the present invention are compared with Last Frame, row, 2d CNN, and 3d CNN models.
TABLE 2 CSI score comparison
TABLE 3 HSS score comparison
TABLE 4 comparison of B-MAE and B-MSE indices
Specifically, the model is predicted and evaluated by using common rainfall prediction and evaluation indexes, which are respectively as follows: CSI (Critical Success Index), HSS (Heidke Skill Score, haddock Skill Score). To evaluate the skill score for a predicted performance that is greater than a certain rainfall intensity threshold (0.5, 2, 5, 10, 30), a value of 1 is assigned when the true rainfall intensity (converting the true radar return reflection value Z to a rainfall intensity value R according to the Z-R relationship defined by equation (5)) or the model predicted rainfall intensity is greater than a certain threshold, and a value of 0 is assigned otherwise. Thus, the number of tp (true positive), the number of fp (false positive), the number of tn (true negative), and the number of fn (false negative) for a specific rainfall intensity can be counted, and the model CSI index value (equation 11) and the HSS index value (equation 12) can be calculated.
From the results table, it can be seen that compared with the traditional radar echo extrapolation methods (Last Frame, ROVER-Linear, ROVER-Nonlinear), the model herein has a significant improvement on CSI and HSS scoring indexes. Compared with the 2D CNN and the 3D CNN based on the non-cyclic neural network architecture model, the model also achieves better prediction performance.
Example two
The embodiment discloses a short-term rainfall forecast device based on a Transformer-IRB, which comprises a processor and a memory, wherein the memory stores a computer program, and the computer program is executed by the processor to realize the steps of the method according to the first embodiment.
EXAMPLE III
The present embodiment discloses a storage medium, which is characterized by storing a computer program, and the computer program is executed by a processor to implement the steps of the method according to the first embodiment.
In summary, the method, the device and the storage medium for forecasting the short rainfall on the basis of the Transformer-IRB have the following beneficial effects: the method combines a self-attention mechanism of a Transformer and an inverted residual error network structure to capture space-time information in a wider range, adopts the inverted residual error network structure with stronger characteristic learning capability to extract complex space-time characteristics in a radar echo diagram sequence, and adopts the self-attention mechanism to capture global space-time dependence in a long range, so that the rainfall evolution process in the long-range space-time range in the radar echo diagram sequence can be better fitted, and the short-rainfall prediction precision is improved.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A method for forecasting short rainfall based on Transformer-IRB, which is characterized by comprising the following steps:
preprocessing a radar echo image;
sending the preprocessed continuous M frames of radar echo images into a pre-constructed and trained short rainfall prediction model based on a transform-IRB to obtain N frames of radar echo images as a short rainfall prediction result, wherein N is larger than M, the short rainfall prediction model comprises a block embedding module, a feature extraction module and a decoding module, the block embedding module extracts the embedding features of continuous multi-frame radar echo images and sends the embedding features into the feature extraction module after adding position codes, the feature extraction module adopts a plurality of transform-inversion residual block modules with multi-head attention layers to form an encoder to carry out feature extraction on the input embedding features and then sends the encoder into the decoding module, and the decoding module decodes the input to obtain the continuous multi-frame radar echo images as the prediction result.
2. The method of claim 1, wherein the preprocessing of the radar echo image comprises: mapping pixel values of the radar echo image to a range of [0,255 ];
the method further comprises the following steps: and acquiring a de-noising mask in advance according to historical data of the radar echo image, and performing point multiplication on the radar echo image output by the short rainfall prediction model and the de-noising mask to obtain a de-noised radar echo image.
3. The method for forecasting short rainfall based on Transformer-IRB as claimed in claim 2, wherein the obtaining of the denoising mask comprises:
defining a denoising mask with the same resolution as the radar echo image, and initializing pixel values of all pixels of the denoising mask to be 1;
searching all pixels of the radar echo map, and recording the pixels of which the Mahalanobis distance of the pixel values is more than 3 times of the standard deviation as noise points;
marking pixels with pixel values larger than 0 and smaller than 70 in the radar echo chart as noise points;
and updating the pixel value of a pixel in the denoising mask, which is the same as the position of the noise point, to be 0.
4. The method of claim 1, wherein the block embedding module comprises two layers of point-by-point convolutional layers and a graph-to-sequence layer, an output of a first convolutional layer is sent to a second convolutional layer after nonlinear mapping is performed on the output of the first convolutional layer by using an activation function, an output of a second convolutional layer is sent to the graph-to-sequence layer after linear mapping is performed on the output of the second convolutional layer, and the graph-to-sequence layer converts a feature graph processed by the second convolutional layer into a feature sequence according to a spatial position.
5. The method of claim 1 or 4, wherein the transform-IRB-based short-term rainfall prediction module comprises a multi-head attention layer, two layer normalization layers, a sequence-to-layer, two point-by-point convolution layers, a depth separable convolution layer, a graph-to-sequence layer;
the transform-inverse residual block module receives the characteristic embedding output by the last module and sends the characteristic embedding into a first layer normalization layer, the output of the first layer normalization layer is sequentially processed by the multi-head attention layer, processed by a second layer normalization layer and sequenced to the layer, the first point-by-point convolution layer is sent into the depth separable convolution layer, the output of the first point-by-point convolution layer and the output of the depth separable convolution layer are added and then sent into the second point-by-point convolution layer, the output of the second point-by-point convolution layer is sent into the sequence layer, and the output of the sequence layer from the graph and the input of the first layer normalization layer are added and then sent into the next module;
the first point-by-point convolution layer and the depth separable convolution layer adopt a GeLU activation function to carry out nonlinear mapping, the second point-by-point convolution layer carries out linear mapping, and the embedded characteristic sequence is converted into a characteristic diagram from the sequence to the layer according to the spatial position.
6. The method for predicting short rainfall reach based on Transformer-IRB as claimed in claim 5, wherein the point-by-point convolution layer is a convolution layer or a deconvolution layer with a grouping number of 1; the depth separable convolutional layer is a convolutional layer or a deconvolution layer with the number of packets as the output dimension.
7. The method for predicting short-term rainfall based on Transformer-IRB as claimed in claim 1, wherein the decoding module is composed of a sequence-to-layer and two deconvolution neural network layers in sequence, and embeds and maps features into a multi-frame radar echo map.
8. The method of claim 1, wherein the method further comprises:
before the short rainfall prediction model is put into use, randomly extracting continuous M + N frames of denoised radar echo images from historical data to randomly train the short rainfall prediction model;
after the short rainfall prediction model is trained randomly and put into use, continuous M + N frames of radar echo maps are extracted from the latest historical data in a sliding mode through a window with the size of M frames for sequential training.
9. A transform-IRB based short-term rainfall prediction device comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, performs the steps of the method according to any one of claims 1 to 8.
10. A storage medium, characterized in that a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111374147.3A CN114137541A (en) | 2021-11-18 | 2021-11-18 | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111374147.3A CN114137541A (en) | 2021-11-18 | 2021-11-18 | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114137541A true CN114137541A (en) | 2022-03-04 |
Family
ID=80390052
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111374147.3A Pending CN114137541A (en) | 2021-11-18 | 2021-11-18 | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114137541A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114594443A (en) * | 2022-03-23 | 2022-06-07 | 南京信息工程大学 | Meteorological radar echo extrapolation method and system based on self-attention mechanism and prediction recurrent neural network |
| CN114764602A (en) * | 2022-05-07 | 2022-07-19 | 东南大学 | Short-term rainfall prediction method based on space-time attention and data fusion |
| CN114818910A (en) * | 2022-04-22 | 2022-07-29 | 北京邮电大学 | Non-contact type blood pressure detection model training method, blood pressure detection method and device |
| CN116051396A (en) * | 2022-11-07 | 2023-05-02 | 中国矿业大学 | Image denoising method based on feature enhancement network and GRU network |
| CN116106856A (en) * | 2023-04-13 | 2023-05-12 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Identification model establishment method and identification method for thunderstorm strong wind and computing equipment |
| CN116229095A (en) * | 2022-12-30 | 2023-06-06 | 北京百度网讯科技有限公司 | Model training method, visual task processing method, device and equipment |
| CN116824372A (en) * | 2023-06-21 | 2023-09-29 | 中国水利水电科学研究院 | Urban rainfall prediction method based on Transformer |
| CN116992231A (en) * | 2023-07-18 | 2023-11-03 | 浙江工业大学 | Short-time rainfall prediction method |
| CN118070846A (en) * | 2024-04-18 | 2024-05-24 | 南京信息工程大学 | Short-term precipitation prediction method, device and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158068A (en) * | 2019-12-31 | 2020-05-15 | 哈尔滨工业大学（深圳） | Short-term prediction method and system based on simple convolutional recurrent neural network |
| CN112907479A (en) * | 2021-03-05 | 2021-06-04 | 西安电子科技大学 | Residual single image rain removing method based on attention mechanism |
| KR102261754B1 (en) * | 2020-05-12 | 2021-06-07 | 한국과학기술원 | Bat-inspired three dimensional imaging method and apparatus |
| CN113162667A (en) * | 2021-04-28 | 2021-07-23 | 中山大学 | Hybrid precoding method, system, medium and communication device |
| CN113496310A (en) * | 2021-06-16 | 2021-10-12 | 国家超级计算深圳中心(深圳云计算中心) | Atmospheric pollutant prediction method and system based on deep learning model |
| CN113610329A (en) * | 2021-10-08 | 2021-11-05 | 南京信息工程大学 | Short-time rainfall approaching forecasting method of double-current convolution long-short term memory network |
| CN113642390A (en) * | 2021-07-06 | 2021-11-12 | 西安理工大学 | Street view image semantic segmentation method based on local attention network |
-
2021
- 2021-11-18 CN CN202111374147.3A patent/CN114137541A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158068A (en) * | 2019-12-31 | 2020-05-15 | 哈尔滨工业大学（深圳） | Short-term prediction method and system based on simple convolutional recurrent neural network |
| KR102261754B1 (en) * | 2020-05-12 | 2021-06-07 | 한국과학기술원 | Bat-inspired three dimensional imaging method and apparatus |
| CN112907479A (en) * | 2021-03-05 | 2021-06-04 | 西安电子科技大学 | Residual single image rain removing method based on attention mechanism |
| CN113162667A (en) * | 2021-04-28 | 2021-07-23 | 中山大学 | Hybrid precoding method, system, medium and communication device |
| CN113496310A (en) * | 2021-06-16 | 2021-10-12 | 国家超级计算深圳中心(深圳云计算中心) | Atmospheric pollutant prediction method and system based on deep learning model |
| CN113642390A (en) * | 2021-07-06 | 2021-11-12 | 西安理工大学 | Street view image semantic segmentation method based on local attention network |
| CN113610329A (en) * | 2021-10-08 | 2021-11-05 | 南京信息工程大学 | Short-time rainfall approaching forecasting method of double-current convolution long-short term memory network |
Non-Patent Citations (2)
| Title |
|---|
| 安东尼奥: "深度学习实战 基于TENSORFLOW 2和KERAS 原书第2版", 31 August 2021, 机械工业出版社, pages: 261 - 265 * |
| 李健兵等: "分布式软目标雷达特性与感知技术述评", 雷达学报, vol. 10, no. 1, 16 June 2020 (2020-06-16), pages 86 - 99 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114594443A (en) * | 2022-03-23 | 2022-06-07 | 南京信息工程大学 | Meteorological radar echo extrapolation method and system based on self-attention mechanism and prediction recurrent neural network |
| CN114818910A (en) * | 2022-04-22 | 2022-07-29 | 北京邮电大学 | Non-contact type blood pressure detection model training method, blood pressure detection method and device |
| CN114764602A (en) * | 2022-05-07 | 2022-07-19 | 东南大学 | Short-term rainfall prediction method based on space-time attention and data fusion |
| CN116051396A (en) * | 2022-11-07 | 2023-05-02 | 中国矿业大学 | Image denoising method based on feature enhancement network and GRU network |
| CN116051396B (en) * | 2022-11-07 | 2023-07-21 | 中国矿业大学 | Image denoising method based on feature enhancement network and GRU network |
| CN116229095A (en) * | 2022-12-30 | 2023-06-06 | 北京百度网讯科技有限公司 | Model training method, visual task processing method, device and equipment |
| CN116106856A (en) * | 2023-04-13 | 2023-05-12 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Identification model establishment method and identification method for thunderstorm strong wind and computing equipment |
| CN116106856B (en) * | 2023-04-13 | 2023-08-18 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Identification model establishment method and identification method for thunderstorm strong wind and computing equipment |
| CN116824372A (en) * | 2023-06-21 | 2023-09-29 | 中国水利水电科学研究院 | Urban rainfall prediction method based on Transformer |
| CN116824372B (en) * | 2023-06-21 | 2023-12-08 | 中国水利水电科学研究院 | Urban rainfall prediction method based on Transformer |
| CN116992231A (en) * | 2023-07-18 | 2023-11-03 | 浙江工业大学 | Short-time rainfall prediction method |
| CN118070846A (en) * | 2024-04-18 | 2024-05-24 | 南京信息工程大学 | Short-term precipitation prediction method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114137541A (en) | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB | |
| GB2547816B (en) | Actually-measured marine environment data assimilation method based on sequence recursive filtering three-dimensional variation | |
| CN110097528A (en) | A kind of image interfusion method based on joint convolution autoencoder network | |
| CN111521990B (en) | Rainfall analysis method based on multilayer radar echo data | |
| CN113361373A (en) | Real-time semantic segmentation method for aerial image in agricultural scene | |
| CN112258625B (en) | Method and system for reconstructing single image to three-dimensional point cloud model based on attention mechanism | |
| CN112215129A (en) | Crowd counting method and system based on sequencing loss and double-branch network | |
| CN115952924B (en) | Short-term wind power prediction method based on optimized VMD and characteristic thermodynamic diagram | |
| CN110807744B (en) | Image defogging method based on convolutional neural network | |
| CN113112583B (en) | 3D human body reconstruction method based on infrared thermal imaging | |
| CN113763539B (en) | Implicit function three-dimensional reconstruction method based on image and three-dimensional input | |
| CN112464172A (en) | Growth parameter active and passive remote sensing inversion method and device | |
| CN117078753A (en) | Progressive feature distribution sampling 6D pose estimation method and system based on camera | |
| CN114973136A (en) | Scene image recognition method under extreme conditions | |
| CN113761395A (en) | Trajectory generation model training method, trajectory generation method and apparatus | |
| CN117765258A (en) | Large-scale point cloud semantic segmentation method based on density self-adaption and attention mechanism | |
| CN117612025A (en) | Remote sensing image roof recognition method and system based on diffusion model | |
| CN116883477A (en) | Monocular depth estimation method | |
| CN117132651A (en) | Three-dimensional human body posture estimation method integrating color image and depth image | |
| CN116563524A (en) | Glance path prediction method based on multi-vision memory unit | |
| CN115965836A (en) | Human behavior posture video data amplification system and method with controllable semantics | |
| CN115331112A (en) | Infrared and visible light image fusion method and system based on multi-granularity word elements | |
| CN113658323A (en) | Method for reconstructing network model in three dimensions based on graph convolution and computer equipment | |
| CN112435321A (en) | Leap Motion hand skeleton Motion data optimization method | |
| CN116958958B (en) | Self-adaptive class-level object attitude estimation method based on graph convolution double-flow shape prior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |