CN117391139A - Weather phenomenon prediction correction method based on improved UNet neural network - Google Patents

Weather phenomenon prediction correction method based on improved UNet neural network Download PDF

Info

Publication number
CN117391139A
CN117391139A CN202311465826.0A CN202311465826A CN117391139A CN 117391139 A CN117391139 A CN 117391139A CN 202311465826 A CN202311465826 A CN 202311465826A CN 117391139 A CN117391139 A CN 117391139A
Authority
CN
China
Prior art keywords
data
neural network
weather phenomenon
sample
unet neural
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311465826.0A
Other languages
Chinese (zh)
Inventor
向婕
闫涵
刘佳鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sprixin Technology Co ltd
Original Assignee
Sprixin Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sprixin Technology Co ltd filed Critical Sprixin Technology Co ltd
Priority to CN202311465826.0A priority Critical patent/CN117391139A/en
Publication of CN117391139A publication Critical patent/CN117391139A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • G06N3/0455Auto-encoder networks; Encoder-decoder networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/0464Convolutional networks [CNN, ConvNet]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/26Government or public services

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Development Economics (AREA)
  • Biophysics (AREA)
  • General Business, Economics & Management (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Educational Administration (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a weather phenomenon prediction correction method based on an improved UNet neural network, which relates to the technical field of weather, and comprises the following steps: acquiring target weather phenomenon forecast data; inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data; the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result after the upper layer of network of the initial decoder in the initial UNet neural network is sampled, so as to obtain an improved decoder. The invention can reduce the deviation of the numerical mode forecasting result.

Description

Weather phenomenon prediction correction method based on improved UNet neural network
Technical Field
The invention relates to the technical field of weather, in particular to a weather phenomenon prediction correction method based on an improved UNet neural network.
Background
With the rapid development of new energy industry, the proportion and total amount of photovoltaic power generation in the whole power system are gradually increased. In some dense areas of the photovoltaic industry, the dense areas are seriously influenced by sand and dust, and the sand and dust influence the power generation condition of the photovoltaic module by influencing the transmission process of atmospheric radiation, so that the safe and stable operation of power grid dispatching is not facilitated. After the sand and dust weather, the sand and dust accumulated on the surface of the photovoltaic panel can reduce the transmissivity of the surface of the photovoltaic module, so that the power generation efficiency of the photovoltaic module is reduced. Therefore, means such as numerical weather forecast and the like are needed to accurately forecast the influence range of sand and dust and the transit time.
The existing sand and dust forecasting technology mainly comprises the following steps: numerical weather forecast patterns, statistical methods, and weather analysis based on weather processes.
The numerical weather forecast mode is to describe the atmospheric motion by using the physical equations of fluid mechanics, thermodynamics and the like, and perform numerical simulation and prediction on a sand and dust sand forming mechanism and a diffusion transmission process; however, the numerical weather forecast mode is limited by the imperfection of a sand and dust power model and a transmission model, the sand and dust sand forming mechanism and the transmission and diffusion process thereof, particularly the influence range and the arrival time of sand and dust are difficult to accurately describe, and the sand and dust concentration forecast has larger errors in time and space distribution, and the improvement of the numerical weather forecast mode needs to create better models and algorithms for expressing the sand and dust forming and transmission mechanism on the basis of establishing better synchronous assimilation of atmospheric components and meteorological observation.
The statistical method mainly utilizes historical data to establish a sand weather prediction model, and establishes a regression relation with sand concentration through conventional meteorological parameters such as temperature, humidity, wind speed, air pressure and the like, and common methods comprise Logistic regression, decision trees, random forests and the like. The statistical method needs to establish a sand prediction model by means of historical actual measurement data, the actual measurement data collection difficulty is high, and the problem that the number of trainable samples (high-concentration sand weather) is small exists. The sand and dust prediction model is built according to the space discrete meteorological site observation data, so that the prediction of the sand and dust concentration space-time distribution cannot be realized, and the model cannot be generalized to new meteorological data points due to the limitation of training data and the deviation in the model training process, so that the model has no better generalization capability.
Because the generation and development of the sand dust are closely related to the change of a large-scale weather system, weather analysis mainly analyzes the evolution rule of weather systems such as cold tides, strong winds and the like, and predicts the generation, development range and duration of the sand dust based on measured meteorological data. The implementation of sand and dust prediction according to weather analysis needs complete expertise and experience, the threshold is higher, and quantitative prediction of sand and dust concentration space-time distribution characteristics is difficult to realize.
It can be seen that the following problems exist in the prior art: the description of the numerical weather forecast mode on the sand and dust transmission process is incomplete, and the model trained by the statistical method cannot keep the space-time continuity of sand and dust transmission and has poor spatial generalization capability.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a weather phenomenon prediction correction method based on an improved UNet neural network.
The invention provides a weather phenomenon prediction correction method based on an improved UNet neural network, which comprises the following steps:
acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode;
inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network;
the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
According to the weather phenomenon prediction correction method based on the improved UNet neural network, the improved UNet neural network is obtained based on the following steps:
acquiring a data set I, wherein the data set I comprises target weather phenomenon forecast sample data input by a model and target weather phenomenon analysis sample data or satellite observation/ground observation sample data output by the model;
performing space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set II;
carrying out space data random segmentation on the second data set to obtain a third training set;
importance sampling is carried out on the data set III according to the target weather phenomenon concentration of the sample grid points in the data set III, so as to form a sampled training set;
training of the improved UNet neural network is performed using the sampled training set.
According to the weather phenomenon prediction correction method based on the improved UNet neural network, provided by the invention, importance sampling is carried out on samples in the data set III according to the target weather phenomenon concentration of sample grid points in the data set III, so as to obtain a sampled training set, and the method comprises the following steps:
Calculating the sampling probability of the sample in the data set III by adopting a formula (1) according to the target weather phenomenon concentration of the sample grid points in the data set III;
wherein p is i For the sampling probability, p, of sample i in the dataset three min For the minimum probability that the sample i is sampled, m is a regulating factor of a data sampling rate, T is a time range, h is the number of grid points of the sample i in the vertical direction, w is the number of grid points of the sample i in the horizontal direction,is saturated value (i.e.)>y i,c S is a saturation constant for the target weather phenomenon concentration of the c-th grid point in the sample i;
based on the sampling probability of the samples in the data set III, uniformly and randomly shifting the samples with high sampling probability in the horizontal direction and the vertical direction respectively to obtain a sampled training set.
According to the weather phenomenon prediction correcting method based on the improved UNet neural network, the method further comprises the following steps:
calculating a loss function by adopting a formula (2);
loss(x i ,y i )=(ssim(x i ,y i )+mse(x i ,y i ) Formula (2);
wherein loss (x i ,y i ) As a loss function, ssim (x i ,y i ) Input data x for sample i in the sampled training set i And output data y i Is a structural similarity index of (2); mse (x) i ,y i ) Is x i And y i Is a mean square error of (c).
According to the weather phenomenon prediction correction method based on the improved UNet neural network, the structural similarity index is calculated based on the formula (3):
the x is i And y i The mean square error of (2) is calculated based on equation (4):
wherein,input data x for training set sample i at time t i Average intensity of target weather phenomenon concentration at w×h grid points, ++>Output data y of training set sample i at time t i Average intensity of target weather phenomenon concentration at w×h grid points, ++>T is the time range, h is the number of lattice points of the training set sample i in the vertical direction, w is the number of lattice points of the training set sample i in the horizontal direction, +.>Input data x for training set sample i at time t i Target weather phenomenon concentration of j-th lattice point,/->Output data y of training set sample i at time t i Target weather phenomenon concentration of jth lattice point, C 1 、C 2 Is constant (I)>Input data x for training set sample i at time t i Standard deviation of unbiased estimation at w×h lattice points, +.>Output data y of training set sample i at time t i Standard deviation of unbiased estimation at w×h lattice points, +.>Input data x for training set sample i at time t i And y i Covariance of unbiased estimates at w x h grid points,
according to the weather phenomenon prediction correction method based on the improved UNet neural network provided by the invention, the matching of space-time dimensions is carried out on the target weather phenomenon prediction sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set II, and the method comprises the following steps:
Carrying out spatial up-sampling and time interpolation on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain target weather phenomenon forecast sample data and target weather phenomenon analysis sample data with consistent space-time resolution;
and carrying out space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or the satellite observation/ground observation sample data with consistent space-time resolution to obtain a data set II.
The invention also provides a weather phenomenon prediction correcting device based on the improved UNet neural network, which comprises the following steps:
the acquisition module is used for acquiring target weather phenomenon forecast data determined based on the numerical weather forecast mode;
the data correction module is used for inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network;
the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
The invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the weather phenomenon forecast correction method based on the improved UNet neural network when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a weather phenomenon prediction correction method based on an improved UNet neural network as described in any one of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a weather phenomenon prediction correction method based on an improved UNet neural network as described in any one of the above.
According to the weather phenomenon prediction correction method based on the improved UNet neural network, the target weather phenomenon prediction data determined based on the numerical weather prediction mode is obtained, the target weather phenomenon prediction data are input into the improved UNet neural network, and corrected target weather phenomenon prediction data output by the improved UNet neural network are obtained, wherein the improved UNet neural network is obtained by performing the following improvement on the basis of the initial UNet neural network: the CBAM attention mechanism is added after each layer of network convolution layer of the initial encoder in the initial UNet neural network, relatively important characteristics on the channel scale (i.e. the time scale) and the space scale can be amplified, the full convolution operation in the initial UNet neural network is replaced by the depth separable convolution operation, training parameters in the model training process are reduced, the incompleteness of the numerical weather forecast mode on the sand dust transmission process description can be improved, and the deviation of the numerical mode forecast result is reduced.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a weather phenomenon prediction correction method based on an improved UNet neural network provided by the invention;
fig. 2 is a schematic structural diagram of a weather phenomenon prediction correcting device based on an improved UNet neural network provided by the invention;
fig. 3 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The weather phenomenon prediction correcting method based on the improved UNet neural network provided by the invention is described below with reference to the attached drawings.
Fig. 1 is a flow chart of a weather phenomenon prediction correction method based on an improved UNet neural network according to the present invention, as shown in fig. 1, the method includes steps 101 to 102, wherein:
and step 101, acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode.
It should be noted that, the target weather phenomenon may include: dust, irradiance, wind field, snow, cloud cover, haze and the like can influence the object of the atmospheric radiation transmission process. The weather phenomenon prediction correction method based on the improved UNet neural network provided by the invention can be applied to a scene in which the influence range and the transit time of the target weather phenomenon such as sand dust, irradiance, wind field, snow, cloud cover and/or haze are required to be accurately predicted.
The execution subject of the method can be a weather phenomenon prediction correcting device based on the improved UNet neural network, such as an electronic device, a server or a control module in the device for executing the weather phenomenon prediction correcting method based on the improved UNet neural network.
Optionally, taking the example that the target weather phenomenon includes sand, the target weather phenomenon numerical weather forecast mode is also referred to as a sand numerical weather forecast mode. The target weather phenomenon forecast data is determined based on a numerical weather forecast model.
102, inputting target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network; the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a attention module (CBAM) attention mechanism of a convolution block after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of an initial decoder in the initial UNet neural network to obtain an improved decoder.
Optionally, the improved UNet neural network is used to correct the dust forecast data.
Taking an initial UNet neural network as an original UNet neural network as an example, the following description will be given: improved UNet neural network training data with convolution block attention module (Convolutional Block Attention Module, CBAM) with depth separable convolution is used. On the basis of the original UNet neural network, a CBAM attention mechanism is added after the convolution operation of each layer of network of the encoder part, and the full convolution operation in the original UNet is replaced by the depth separable convolution. The network structure of the decoder part is to splice the output of the attention adding mechanism in the same layer network of the encoder with the sampled result of the upper layer network of the decoder.
Alternatively, UNet mainly uses convolution, pooling, deconvolution and other modules to realize encoder feature extraction and decoder image size recovery, and by providing a depth-separable convolution CBAM attention module to improve the recognition capability of important features in time and space, the encoding-decoding structure design and jump-type connection structure have application in UNet models and more variant models thereof, and can be replaced by similar models, such as RES-UNet, CU-net and the like, and can also be replaced by neural network models such as GAN, GNN and the like.
According to the weather phenomenon prediction correction method based on the improved UNet neural network, the target weather phenomenon prediction data determined based on the numerical weather prediction mode is obtained, the target weather phenomenon prediction data are input into the improved UNet neural network, and corrected target weather phenomenon prediction data output by the improved UNet neural network are obtained, wherein the improved UNet neural network is obtained by performing the following improvement on the basis of the initial UNet neural network: the CBAM attention mechanism is added after each layer of network convolution layer of the initial encoder in the initial UNet neural network, relatively important characteristics on the channel scale (i.e. the time scale) and the space scale can be amplified, the full convolution operation in the initial UNet neural network is replaced by the depth separable convolution operation, training parameters in the model training process are reduced, the incompleteness of the numerical weather forecast mode on the sand dust transmission process description can be improved, and the deviation of the numerical mode forecast result is reduced.
Optionally, the improved UNet neural network is obtained based on the following steps a-e:
step a, acquiring a data set I, wherein the data set I comprises target weather phenomenon forecast sample data input by a model and target weather phenomenon analysis sample data or satellite observation/ground observation sample data output by the model.
Specifically, target weather phenomenon forecast sample data, such as the gothic atmospheric monitoring service (Copernicus Atmosphere Monitoring Service, CAMS) global atmospheric composition forecast (global atmospheric composition forecasts) sand forecast dataset X from the mid-european weather forecast center (European Centre for Medium-Range Weather Forecasts, ECMWF), or grid point forecast data of numerical weather forecast versus irradiance, wind field, snow, cloud cover, or haze.
The target weather phenomenon is analyzed to sample data, such as a sand and dust analysis data set Y from CAMS global reanalysis (EAC 4), or ECMWF fifth generation atmospheric analysis data set (ECMWF Reanalysis v, ERA 5), and other day-based measured data sets of satellite remote sensing inversion, or other fusion data sets that can characterize measured irradiance, wind field, snow, cloud cover, and haze levels. The target weather phenomenon analysis sample data includes sand aerosol optical thickness, sand mass mixing ratio, and contaminant concentration data.
And b, performing space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set II.
Specifically, spatial up-sampling and time interpolation are performed on target weather phenomenon forecast sample data and target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain target weather phenomenon forecast sample data and target weather phenomenon analysis sample data or satellite observation/ground observation sample data with consistent space-time resolution, and space-time dimension matching is performed on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data with consistent space-time resolution to obtain a second data set. For example, the input data set is Xn and the output data set is Yn. The single data set sample comprises input data Xi and output data Yi, the dimensions of the input data Xi and the output data Yi of the single data set sample are T, H and W, T is a time range, H is the number of grid points in the vertical direction of the single data set, and W is the number of grid points in the horizontal direction of the single data set.
And c, carrying out space data random segmentation on the data set II to obtain a data set III.
Optionally, the samples in the multiple data sets three are, for example, (xn, yn), and the dimensions of the input xi and the output yi of the single training set are t×h×w, where h is the number of grid points in the vertical direction of the sample of the single training set, and w is the number of grid points in the horizontal direction of the sample of the single training set, and h=w, so that the number of training sets is increased, and the occupation of single iteration video memory during training is reduced.
And d, sampling importance of the samples in the data set three according to the target weather phenomenon concentration of each grid point of the samples in the data set three, and obtaining a sampled training set.
Optionally, importance sampling is used to increase the occurrence frequency of high-concentration sand weather in the training set, so that the problem of insufficient sample size of the sand weather in the training set is solved.
Specifically, firstly, calculating sampling probability of each sample in the data set three by adopting a formula (1) according to target weather phenomenon concentration of grid points in each sample in the data set three;
wherein p is i For the sampling probability, p, of sample i in the training set min For the minimum probability of the training set sample i being sampled, m is a regulating factor of a data sampling rate, T is a time range, h is the number of grid points of the training set sample i in the vertical direction, w is the number of grid points of the training set sample i in the horizontal direction, Is saturated value (i.e.)>y i,c The target weather phenomenon concentration of the c-th grid point in the training set sample i is obtained, and s is a saturation constant; this approach tends to be data for the presence of high concentrations of sand.
Then, based on the sampling probability of each sample in the data set III, uniformly and randomly shifting samples with high sampling probability in the horizontal direction and the vertical direction respectively to form a sampled training set.
And e, training the improved UNet neural network by using the sampled training set.
Alternatively, the loss function may employ an average weighting using a structural similarity index and a mean square error, MAE, mae+mse, SSIM loss weighting, and the like, which is not limited by the present invention.
Taking the average weighting of the structural similarity index and the mean square error as an example, calculating the loss function by adopting a formula (2);
loss(x i ,y i )=(ssim(x i ,y i )+mse(x i ,y i ) Formula (2);
wherein loss (x i ,y i ) As a loss function, ssim (x i ,y i ) Input data x for sample i in training set i And output data y i Is a structural similarity index of (2); mse (x) i ,y i ) Is x i And y i Is a mean square error of (c).
Taking the example that the target weather phenomenon is sand. The structural similarity index is mainly used for evaluating the prediction result through the output of model prediction and the characteristics of three aspects of brightness, contrast and structure of analysis sand and dust data spatial distribution map, so that the problem of reduced spatial effective resolution of the prediction result of the UNet neural network model can be solved to a certain extent, and the description of sand and dust concentration spatial distribution details is obviously enhanced.
The structural similarity index is calculated based on the formula (3):
the x is i And y i The mean square error of (2) is calculated based on equation (4):
wherein,input data x for training set sample i at time t i Target weather phenomenon concentration at w×h grid pointsAverage intensity,/->Output data y of training set sample i at time t i Average intensity of target weather phenomenon concentration at w×h grid points, ++>T is the time range, h is the number of lattice points of the training set sample i in the vertical direction, w is the number of lattice points of the training set sample i in the horizontal direction, +.>Input data x for training set sample i at time t i Target weather phenomenon concentration of jth grid point,/-j>Output data y of training set sample i at time t i Target weather phenomenon concentration of the j-th grid point in (3); c (C) 1 、C 2 As a constant, use C 1 And C 2 The fact that ssim tends to infinity when the denominator approaches 0 in the calculation process can be avoided; />Input data x for training set sample i at time t i Is the standard deviation of unbiased estimation of w.h lattice points,/for>Output data y of training set sample i at time t i Is the standard deviation of unbiased estimation of w.h lattice points,/for>Input data x for training set sample i at time t i And y i Covariance of unbiased estimation at w×h lattice points, use +. >As a measure of contrast, use is made of/>As a measure of the comparison of the results,
according to the weather phenomenon prediction correction method based on the improved UNet neural network, which is provided by the invention, the change characteristics of the sand concentration space distribution in the adjacent time are identified by using a CBAM attention mechanism with depth separable convolution; the original full convolution in UNet is replaced by the convolution operation with separable depth, so that training parameters in the training process of the sand prediction space-time correction model are reduced; based on data segmentation and importance sampling, training data are enhanced, and the number of high-concentration sand and dust samples in a training set is increased; and using the weighted results of the structural similarity index and the mean square error of the prediction output and analysis data as a loss function, improving the problem of reduced effective resolution of the model prediction output space, and reducing the error of the sand concentration space-time distribution. The weather phenomenon prediction correction method based on the improved UNet neural network is suitable for being applied to the quantities of directly or indirectly influencing the atmospheric radiation transmission process aiming at sand dust, irradiance, wind fields, snow cover, cloud cover, haze and the like.
Examples are as follows: an improved UNet model with attention mechanisms is trained and optimized using CAMS dust predictions and analysis datasets over a geographic area, taking the influence of three large-scale dust weather as an example. Compared with the conventional ConvLSTM correction CAMS sand dust mode prediction result, the prediction result based on the weather phenomenon prediction correction method based on the improved UNet neural network provided by the invention has the advantages that the CSI with the optical thickness of the sand dust aerosol exceeding 1 is improved by 4.5%, the CSI with the optical thickness of the sand dust aerosol exceeding 0.5 is improved by 2%, and the optimization of the space-time distribution characteristic of sand dust prediction concentration and the space range and arrival time of sand dust invasion is realized.
The weather phenomenon prediction correcting device based on the improved UNet neural network provided by the invention is described below, and the weather phenomenon prediction correcting device based on the improved UNet neural network described below and the weather phenomenon prediction correcting method based on the improved UNet neural network described above can be correspondingly referred to each other.
Fig. 2 is a schematic structural diagram of a weather phenomenon prediction correcting device based on an improved UNet neural network provided by the invention; the weather phenomenon prediction correcting apparatus 200 based on the improved UNet neural network includes: an acquisition module 201 and a data correction module 202; wherein,
an acquisition module 201, configured to acquire target weather phenomenon prediction data determined based on a numerical weather prediction mode;
the data correction module 202 is configured to input the target weather phenomenon prediction data to an improved UNet neural network, and obtain corrected target weather phenomenon prediction data output by the improved UNet neural network;
the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
According to the weather phenomenon prediction correcting device based on the improved UNet neural network, the target weather phenomenon prediction data determined based on the numerical weather prediction mode is obtained, the target weather phenomenon prediction data are input into the improved UNet neural network, and corrected target weather phenomenon prediction data output by the improved UNet neural network are obtained, wherein the improved UNet neural network is obtained by performing the following improvement on the basis of the initial UNet neural network: the CBAM attention mechanism is added after each layer of network convolution layer of the initial encoder in the initial UNet neural network, relatively important characteristics on the channel scale (i.e. the time scale) and the space scale can be amplified, the full convolution operation in the initial UNet neural network is replaced by the depth separable convolution operation, training parameters in the model training process are reduced, the incompleteness of the numerical weather forecast mode on the sand dust transmission process description can be improved, and the deviation of the numerical mode forecast result is reduced.
Optionally, the improved UNet neural network is obtained based on the following steps:
acquiring a data set I, wherein the data set I comprises target weather phenomenon forecast sample data input by a model and target weather phenomenon analysis sample data or satellite observation/ground observation sample data output by the model;
Performing space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set II;
carrying out space data random segmentation on the data set II to obtain a data set III;
importance sampling is carried out on the data sets three according to the target weather phenomenon concentration of each grid point of the samples in the data sets three, and a sampled training set is obtained;
training of the improved UNet neural network is performed using the sampled training set.
Optionally, the importance sampling is performed on each third data set according to the target weather phenomenon concentration of each grid point in each third data set to obtain a sampled training set, which includes:
calculating the sampling probability of the samples in the data sets three by adopting a formula (1) according to the target weather phenomenon concentration of each grid point of the samples in the data sets three;
wherein p is i Sampling probability, p, for samples i in the data set three to be sampled to a training set min For the minimum probability that the sample i in the training set is sampled, m is a regulating factor of a data sampling rate, T is a time range, h is the number of grid points of the sample i in the training set in the vertical direction, w is the number of grid points of the sample i in the horizontal direction, Is a saturation value, and is set to be a saturation value,y i,c the target weather phenomenon concentration of the c-th grid point in the training set sample i is obtained, and s is a saturation constant;
based on the sampling probability of the samples in the data set III, uniformly and randomly shifting the samples with high sampling probability in the horizontal direction and the vertical direction respectively to obtain a sampled training set.
Optionally, calculating a loss function using equation (2);
loss(x i ,y i )=(ssim(x i ,y i )+mse(x i ,y i ) Formula (2);
wherein loss (x i ,y i ) As a loss function, ssim (x i ,y i ) Input data x for sample i in the sampled training set i And output data y i Is a structural similarity index of (2); mse (x) i ,y i ) Is x i And y i Is a mean square error of (c).
Optionally, the structural similarity index is calculated based on formula (3):
the x is i And y i The mean square error of (2) is calculated based on equation (4):
wherein,input data x for training set sample i at time t i W.h grid points, mean intensity of target weather phenomenon concentration, +.>Output data y of training set sample i at time t i W.h grid points, mean intensity of target weather phenomenon concentration, +.>T is the time range, h is the number of lattice points of the sample i in the training set in the vertical direction, w is the number of lattice points of the sample i in the training set in the horizontal direction, +.>Input data x for sample i in training set at time t i Target weather phenomenon concentration of j-th lattice point,/->Output data y of sample i in training set for time t i Target weather phenomenon concentration of jth lattice point, C 1 、C 2 Is constant (I)>Input data x for sample i in training set at time t i Is the standard deviation of unbiased estimation of w.h lattice points,/for>Output data y of training set sample i at time t i Is the standard deviation of unbiased estimation of w.h lattice points,/for>Input data x for training set sample i at time t i And y i Covariance of unbiased estimates of the respective w x h grid points,
optionally, the performing space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set two includes:
performing spatial up-sampling and time interpolation on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or the satellite observation/ground observation sample data to obtain target weather phenomenon forecast sample data and target weather phenomenon analysis sample data or satellite observation/ground observation sample data with consistent space-time resolution;
and carrying out space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or the satellite observation/ground observation sample data with consistent space-time resolution to obtain a data set II.
Fig. 3 illustrates a physical schematic diagram of an electronic device, as shown in fig. 3, where the electronic device may include: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a weather phenomenon forecast correction method based on the modified UNet neural network, the method comprising: acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode; inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network; the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing the weather phenomenon prediction correction method based on the improved UNet neural network provided by the above methods, the method comprising: acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode; inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network; the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the weather phenomenon prediction correction method based on the improved UNet neural network provided by the above methods, the method comprising: acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode; inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network; the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (9)

1. A weather phenomenon prediction correction method based on an improved UNet neural network, the method comprising:
acquiring target weather phenomenon forecast data determined based on a numerical weather forecast mode;
inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network;
the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
2. The weather phenomenon prediction correction method based on the improved UNet neural network according to claim 1, wherein the improved UNet neural network is obtained based on the steps of:
acquiring a data set I, wherein the data set I comprises target weather phenomenon forecast sample data input by a model and target weather phenomenon analysis sample data or satellite observation/ground observation sample data output by the model;
Performing space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or satellite observation/ground observation sample data to obtain a data set II;
carrying out space data random segmentation on the data set II to obtain a data set III;
importance sampling is carried out on the data set III according to the target weather phenomenon concentration of the sample grid points in the data set III, so as to form a sampled training set;
training of the improved UNet neural network is performed using the sampled training set.
3. The weather phenomenon prediction correction method based on the improved UNet neural network according to claim 2, wherein the importance sampling is performed on the samples in the data set three according to the target weather phenomenon concentration of the sample grid points in the data set three, so as to obtain a sampled training set, and the method comprises the following steps:
calculating the sampling probability of the sample in the data set III by adopting a formula (1) according to the target weather phenomenon concentration of the sample grid points in the data set III;
wherein p is i Sampling probability, p, for samples i in the data set three to be sampled to a training set min For the training setThe minimum probability of sampling the sample i, m is a regulating factor of the data sampling rate, T is a time range, h is the number of grid points of the sample i in the vertical direction in the training set, w is the number of grid points of the sample i in the horizontal direction in the training set, Is a saturation value, and is set to be a saturation value,y i,c the target weather phenomenon concentration of the c-th grid point of the sample i in the training set is obtained, and s is a saturation constant;
based on the sampling probability of the data set III, uniformly and randomly shifting samples with high sampling probability in the horizontal direction and the vertical direction respectively to obtain a sampled training set.
4. The improved UNet neural network-based weather phenomenon prediction correction method according to claim 2, further comprising:
calculating a loss function by adopting a formula (2);
loss(x i ,y i )=(ssim(x i ,y i )+mse(x i ,y i ) Formula (2);
wherein loss (x i ,y i ) As a loss function, ssim (x i ,y i ) Input data x for sample i in training set i And output data y i Is a structural similarity index of (2); mse (x) i ,y i ) Is x i And y i Is a mean square error of (c).
5. The improved UNet neural network-based weather phenomenon prediction correction method according to claim 4, wherein the structural similarity index is calculated based on formula (3):
the x is i And y i The mean square error of (2) is calculated based on equation (4):
wherein,input data x for training set sample i at time t i Average intensity of target weather phenomenon concentration at w×h grid points, ++>Output data y of sample i in training set for time t i Average intensity of target weather phenomenon concentration at w×h grid points, ++ >T is the time range, h is the number of lattice points of the sample i in the training set in the vertical direction, w is the number of lattice points of the sample i in the training set in the horizontal direction, +.>Input data x for sample i in training set at time t i Target weather phenomenon concentration of jth lattice point,/->Output data y of sample i in training set for time t i Target weather phenomenon concentration of jth lattice point, C 1 、C 2 Is constant (I)>Input data x for sample i in training set at time t i Standard deviation of unbiased estimation at w×h lattice points, ++>Output data y of sample i in training set for time t i Standard deviation of unbiased estimation at w×h lattice points, ++>Input data x for sample i in training set at time t i And output data y i Covariance of unbiased estimates at corresponding w x h grid points,
6. the method for correcting weather phenomenon forecast based on improved UNet neural network according to claim 2, wherein the matching the target weather phenomenon forecast sample data with the target weather phenomenon analysis sample data or satellite observation/ground observation sample data in space-time dimension to obtain a data set two includes:
performing spatial up-sampling and time interpolation on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or the satellite observation/ground observation sample data to obtain target weather phenomenon forecast sample data and target weather phenomenon analysis sample data or satellite observation/ground observation sample data with consistent space-time resolution;
And carrying out space-time dimension matching on the target weather phenomenon forecast sample data and the target weather phenomenon analysis sample data or the satellite observation/ground observation sample data with consistent space-time resolution to obtain a data set II.
7. A weather phenomenon prediction correcting device based on an improved UNet neural network, comprising:
the acquisition module is used for acquiring target weather phenomenon forecast data determined based on the numerical weather forecast mode;
the data correction module is used for inputting the target weather phenomenon forecast data into an improved UNet neural network to obtain corrected target weather phenomenon forecast data output by the improved UNet neural network;
the improved UNet neural network is obtained by carrying out the following improvement on the basis of the initial UNet neural network: adding a CBAM attention mechanism after each layer of network convolution layer of an initial encoder in an initial UNet neural network, replacing full convolution operation in the initial UNet neural network with depth-separable convolution operation, and splicing the output of the attention mechanism added in the same layer of network of the initial encoder with the result sampled by the upper layer of network of the initial decoder in the initial UNet neural network to obtain an improved decoder.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the weather phenomenon prediction correction method based on an improved UNet neural network as claimed in any one of claims 1 to 6 when the program is executed by the processor.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the weather phenomenon prediction correction method based on the modified UNet neural network according to any one of claims 1 to 6.
CN202311465826.0A 2023-11-06 2023-11-06 Weather phenomenon prediction correction method based on improved UNet neural network Pending CN117391139A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311465826.0A CN117391139A (en) 2023-11-06 2023-11-06 Weather phenomenon prediction correction method based on improved UNet neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311465826.0A CN117391139A (en) 2023-11-06 2023-11-06 Weather phenomenon prediction correction method based on improved UNet neural network

Publications (1)

Publication Number Publication Date
CN117391139A true CN117391139A (en) 2024-01-12

Family

ID=89471819

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311465826.0A Pending CN117391139A (en) 2023-11-06 2023-11-06 Weather phenomenon prediction correction method based on improved UNet neural network

Country Status (1)

Country Link
CN (1) CN117391139A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118153640A (en) * 2024-05-13 2024-06-07 南京信息工程大学 Multisource satellite effective wave height fusion method based on improved convolutional neural network

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118153640A (en) * 2024-05-13 2024-06-07 南京信息工程大学 Multisource satellite effective wave height fusion method based on improved convolutional neural network

Similar Documents

Publication Publication Date Title
CN109492830B (en) Mobile pollution source emission concentration prediction method based on time-space deep learning
CN112766549A (en) Air pollutant concentration forecasting method and device and storage medium
CN114926749B (en) Near-surface atmospheric pollutant inversion method and system based on remote sensing image
CN110232471B (en) Rainfall sensor network node layout optimization method and device
CN113610243B (en) Atmospheric pollutant tracing method based on coupled machine learning and correlation analysis
CN111369057A (en) Air quality prediction optimization method and system based on deep learning
CN112884217B (en) Sea surface height forecasting method based on multi-model integration
CN117391139A (en) Weather phenomenon prediction correction method based on improved UNet neural network
CN111598460B (en) Method, device, equipment and storage medium for monitoring heavy metal content of soil
CN115237896B (en) Data preprocessing method and system based on deep learning forecast air quality
CN114445634A (en) Sea wave height prediction method and system based on deep learning model
CN116760031B (en) High-time-resolution photovoltaic power prediction method and device based on meteorological data
CN116449462B (en) Method, system, storage medium and equipment for predicting effective wave height space-time sequence of sea wave
CN114418211A (en) Method and device for correcting precipitation
CN117374956A (en) Short-term prediction method for photovoltaic power generation of comprehensive energy station
Grönquist et al. Predicting weather uncertainty with deep convnets
CN115935809A (en) Photovoltaic power generation power interval prediction method based on cloud cover characteristic of foundation cloud cover
Kong et al. A deep spatio-temporal forecasting model for multi-site weather prediction post-processing
CN115420690A (en) Near-surface trace gas concentration inversion model and inversion method
CN116525135B (en) Method for predicting epidemic situation development situation by space-time model based on meteorological factors
CN110648030A (en) Method and device for predicting seawater temperature
CN116522648A (en) Lake algae state prediction method and application
CN116128168A (en) Weather prediction method based on causal expansion convolution and Autoformer
CN114997490A (en) Construction method, prediction method, device and equipment of temperature profile prediction model
CN113486515B (en) Sub-season-annual scale integrated climate mode set prediction system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination